PJSIP視頻用戶開發

# PJSip
PJSip開發。

#3.2 傳來的通話：

用call setting啓用或拒絕video（pjsua_call_setting，via vid_cnt setting）

API：

pjsua_call_answer2() (so for example, to reject the video, set vid_cnt to 0 and call pjsua_call_answer2()).


但要顯示該視頻，以及傳出視頻，則需要另設。


#3.2.1 自動顯示來的視頻：

pjsua_acc_config cfg;pjsua_acc_config_default(&cfg);cfg.vid_in_auto_show = PJ_TRUE;

#3.2.2 手動設置，顯示或隱藏來的視頻：

1.用pjsua_call_get_vid_stream_idx()來索引出呼叫傳來的媒體流

2.用pjsua_call_info，根據索引定位到流，然後獲得windowID關聯到視頻。

int vid_idx;pjsua_vid_win_id wid;vid_idx = pjsua_call_get_vid_stream_idx(call_id);if (vid_idx >= 0) {    pjsua_call_info ci;    pjsua_call_get_info(call_id, &ci);    wid = ci.media[vid_idx].stream.vid.win_in;}

3.有了windowID，運用 pjsua_vid_win_get_info()來獲得關聯的視頻，然後用本機API來顯示窗口，或用pjsua_vid_win_set_show()


#3.2.3 控制傳來的視頻流：

以上提到的只是在本地改變視頻，但也可以通過pjsua_call_set_vid_strm()來操作對方傳的視頻流。

具有以下操作：（改變媒體方向、關閉媒體流、增加流）

PJSUA_CALL_VID_STRM_CHANGE_DIR: change the media direction (e.g. to "sendonly", or even "inactive")
PJSUA_CALL_VID_STRM_REMOVE: remove the media stream altogether by settings its port to zero
PJSUA_CALL_VID_STRM_ADD: add new video media stream

由於這些操作是導致遠程改變的，所以得過一會才能生效，可以運行回調函數on_call_media_state()來監視狀態。


#3.3 呼出的通話：

#3.3.1

呼出通話前把 pjsua_call_setting.flag 設置成 PJSUA_CALL_INCLUDE_DISABLED_MEDIA ，意味着禁止視頻。

傳出的視頻跟傳入的是分開操作的，而且也不會默認開啓，就算呼入請求是包含視頻的。

由 pjsua_acc_config.vid_out_auto_transmit 來設置傳出視頻，默認是 PJ_FALSE. 設爲 PJ_TRUE 就可以開啓所有呼出通話和帶視頻的呼入通話的視頻傳出。

#3.3.2 默認的捕獲設備：

在 pjsua_acc_config.vid_cap_dev 設置默認捕獲設備，在這裏設好過在後面的所有API裏面設（-_-!神馬意思）。

#3.3.3 控制傳出媒體流：

用 pjsua_call_set_vid_strm()

    PJSUA_CALL_VID_STRM_ADD: add a new video stream
    PJSUA_CALL_VID_STRM_REMOVE: remove video stream (set port to zero)
    PJSUA_CALL_VID_STRM_CHANGE_DIR: change direction or deactivate (i.e. set direction to "inactive")
    PJSUA_CALL_VID_STRM_CHANGE_CAP_DEV: change capture device
    PJSUA_CALL_VID_STRM_START_TRANSMIT: start previously stopped transmission
    PJSUA_CALL_VID_STRM_STOP_TRANSMIT: stop transmission

設置vid_cnt of pjsua_call_setting 可以增刪視頻。。。


#3.5 視頻窗口的工作：

關於視頻窗口：

在呼叫中，傳入的視頻流的視頻窗口是由 pjsua_call_info 結構體控制的

預覽窗口要關聯捕獲設備可以通過查詢 pjsua_vid_preview_get_win()

另外，應用程序也可以利用 pjsua_vid_enum_wins() 來列舉所有視頻窗口

應用程序可以用 pjsua_vid_win_get_info() 獲得 pjsua_vid_win_info 。大多數應用程序會對一個窗口的屬性感興趣，就是本地窗口句柄native window handle。video handle由pjsua_vid_win_info裏面的 pjmedia_vid_dev_hwnd 結構體控制。利用這個video handle，應用程序可以把video window 嵌入到 GUI 。另外，pj庫也提供了一些常用API來操作窗口，諸如pjsua_vid_win_set_show()，pjsua_vid_win_set_size()等。但這些API不擔保有效，因爲這取決於底層設備。

#3.6 修改視頻電話的視頻編解碼器的參數：

在 pjmedia_vid_codec_param 裏面制定視頻編解碼參數。可以分別設置編碼和解碼兩個方向的參數，更改後，用pjsua_vid_codec_set_param()使其生效。

const pj_str_t codec_id = {"H264", 4};pjmedia_vid_codec_param param;pjsua_vid_codec_get_param(&codec_id, &param);/* Modify param here */...pjsua_vid_codec_set_param(&codec_id, &param);

#3.6.1 大小與分辨率：{

    編碼：pjmedia_vid_codec_param.enc_fmt.det.vid.size

    /* Sending 1280 x 720 */

    param.enc_fmt.det.vid.size.w = 1280;

    param.enc_fmt.det.vid.size.h = 720;

    但這可能要根據遠程端的能力來調整，比如，如果遠程信號說他最高支持640*480，而這裏設了1280*720，那麼將會是640*480生效。


    解碼：

    1.把 pjmedia_vid_codec_param.dec_fmt.det.vid.size 設爲最高預期的想要遠方傳來的分辨率大小。

    2.通過fmtp（ pjmedia_vid_codec_param.dec_fmtp ）向遠程端發信號來配置。

            H263-1998, e.g:

        /* 1st preference: 352 x 288 (CIF) */

        param.dec_fmtp.param[n].name = pj_str("CIF");

        /* The value actually specifies framerate, see framerate section below */

        param.dec_fmtp.param[n].val = pj_str("1");

        /* 2nd preference: 176 x 144 (QCIF) */

        param.dec_fmtp.param[n+1].name = pj_str("QCIF");

        /* The value actually specifies framerate, see framerate section below */

        param.dec_fmtp.param[n+1].val = pj_str("1");

            H264, the size is implicitly specified in H264 level (check the standard specification or ?this) and on SDP, the H264 level is signalled via H264 SDP fmtp ?profile-level-id, e.g:

        /* Can receive up to 1280×720 @30fps */

        param.dec_fmtp.param[n].name = pj_str("profile-level-id");

        /* Set the profile level to "1f", which means level 3.1 */

        param.dec_fmtp.param[n].val = pj_str("xxxx1f");

PJSIP Video Users Guide （PJSIP視頻用戶指南） - kai7613 - 天空安靜

         

}

#3.6.2 幀速率：{

    編碼方向：pjmedia_vid_codec_param.enc_fmt.det.vid.fps

    /* Sending @30fps */

    param.enc_fmt.det.vid.fps.num   = 30;

    param.enc_fmt.det.vid.fps.denum = 1;

    如果建立通話前，視頻已經開始預覽，那麼將會使用設備的默認幀速率，並且沿用到通話中，而且不管此處如何設置幀速率，都不會有所改變。解決的方法只能是在建立媒體前先禁用預覽再重新啓用媒體。

    解碼方向：

    1.pjmedia_vid_codec_param.dec_fmt.det.vid.fps設爲預期的最高值。

    2.通過fmtp（ pjmedia_vid_codec_param.dec_fmtp ）向遠程端發信號來配置。

    e.g.同上設置分辨率的部分。

}

#3.6.3 比特率：{

    /* Bitrate range preferred: 512-1024kbps */

    param.enc_fmt.det.vid.avg_bps = 512000;

    param.enc_fmt.det.vid.max_bps = 1024000;

    比特率是雙方向對稱的。

    調整比特率設置其實就是在調整SDP。

    編碼方向的比特率將會受對方在SDP中協定的最大比特率限制。

    如果渴望指定一個比特率，可以通過SDP fmtp手動設置，如下：

    /* H263 specific maximum bitrate 512kbps */

    param.dec_fmtp.param[n].name = pj_str("MaxBR");

    param.dec_fmtp.param[n].val = pj_str("5120"); /* = max_bps / 100 */

}


#5. 視頻的一些API：

5.1 設備枚舉API：

/**獲得系統中的視頻設備數量 * Get the number of video devices installed in the system. *返回：設備數量 * @return        The number of devices. */PJ_DECL(unsigned) pjsua_vid_dev_count(void);

/**根據索引取得設備信息 * Retrieve the video device info for the specified device index. * * @param id        The device index.  設備索引 * @param vdi        Device info to be initialized.  存放信息的結構體 * * @return        PJ_SUCCESS on success, or the appropriate error code. */PJ_DECL(pj_status_t) pjsua_vid_dev_get_info(pjmedia_vid_dev_index id,                                            pjmedia_vid_dev_info *vdi);

/**枚舉系統中所有視頻設備 * Enum all video devices installed in the system. * * @param info        Array of info to be initialized. 存放設備信息的數組 * @param count        On input, specifies max elements in the array. 傳入時，指定數組的最大值 *            On return, it contains actual number of elements   傳回來實際的數量 *            that have been initialized. * * @return        PJ_SUCCESS on success, or the appropriate error code. */PJ_DECL(pj_status_t) pjsua_vid_enum_devs(pjmedia_vid_dev_info info[],                     unsigned *count);

/**刷新系統中的設備列表，這將可能導致設備索引號的重新編排，但不影響已經使用中的媒體流。刷新後，要注意所有的索引pjmedia_vid_dev_index變量已經失效，調用以索引作爲參數的函數就要尤其注意。 * Refresh the list of video devices installed in the system. This function * will only refresh the list of videoo device so all active video streams will * be unaffected. After refreshing the device list, application MUST make sure * to update all index references to video devices (i.e. all variables of type * pjmedia_vid_dev_index) before calling any function that accepts video device * index as its parameter. * * @return        PJ_SUCCESS on successful operation or the appropriate *            error code. */PJ_DECL(pj_status_t) pjmedia_vid_dev_refresh(void);

5.2 視頻預覽API：

要開始視頻預覽前，要先用pjsua_vid_preview_param_default()初始化以下結構體：

/** * Parameters for starting video preview with pjsua_vid_preview_start(). * Application should initialize this structure with * pjsua_vid_preview_param_default(). */typedef struct pjsua_vid_preview_param{    /**視頻渲染器（即播放設備）的ID。     * Device ID for the video renderer to be used for rendering the     * capture stream for preview.     */    pjmedia_vid_dev_index    rend_id;} pjsua_vid_preview_param;

/**爲指定的捕獲設備開啓視頻預覽窗口 * Start video preview window for the specified capture device. * * @param id        The capture device ID where its preview will be *             started. 捕獲設備的ID * @param prm        Optional video preview parameters. Specify NULL *             to use default values.預覽選項，NULL爲默認 * * @return        PJ_SUCCESS on success, or the appropriate error code. */PJ_DECL(pj_status_t) pjsua_vid_preview_start(pjmedia_vid_dev_index id,                                             pjsua_vid_preview_param *prm);

/**如果有的話，獲取與捕獲設備關聯了的預覽窗口handle * Get the preview window handle associated with the capture device, if any. * * @param id        The capture device ID. * * @return        The window ID of the preview window for the *             specified capture device ID, or NULL if preview *             does not exist. */PJ_DECL(pjsua_vid_win_id) pjsua_vid_preview_get_win(pjmedia_vid_dev_index id);

/**停止視頻預覽 * Stop video preview. * * @param id        The capture device ID. * * @return        PJ_SUCCESS on success, or the appropriate error code. */PJ_DECL(pj_status_t) pjsua_vid_preview_stop(pjmedia_vid_dev_index id);

5.3 視頻配置：

視頻啓不啓用在 pjsua_call_setting.vid_cnt中設置，一般爲1即啓用，0爲不啓用。

視頻的大部分設置都在 pjsua_acc_config 的以下域：

pj_bool_t        vid_in_auto_show;  指定傳入的視頻是否自動顯示在屏幕上。 PJ_TRUE / PJ_FALSE

pjsua_call_get_info();  枚舉媒體流

pj_bool_t        vid_out_auto_transmit;  設定當有視頻來電，或撥出電話時，是否默認激活視頻傳出。PJ_TRUE / PJ_FALSE

無論上面這個怎麼設置，都可以用 pjsua_call_set_vid_strm(); 來開始或停止視頻傳出。

pjmedia_vid_dev_index vid_cap_dev;  指定使用的捕獲設備。Default: PJMEDIA_VID_DEFAULT_CAPTURE_DEV

pjmedia_vid_dev_index vid_rend_dev;  指定使用的顯示設備。Default: PJMEDIA_VID_DEFAULT_RENDER_DEV


5.4 視頻通話操作：


除了以上的視頻配置操作，還可以在呼叫時用pjsua_call_set_vid_strm()來配置視頻。

要配置時，首先要利用以下兩個結構體的其中一個：

typedef enum pjsua_call_vid_strm_op{    /**     * No operation     */    PJSUA_CALL_VID_STRM_NO_OP,    /**增加視頻流     * Add a new video stream. This will add a new m=video line to     * the media, regardless of whether existing video is/are present     * or not.  This will cause re-INVITE or UPDATE to be sent to remote     * party.     */    PJSUA_CALL_VID_STRM_ADD,    /**移除視頻流     * Remove/disable an existing video stream. This will     * cause re-INVITE or UPDATE to be sent to remote party.     */    PJSUA_CALL_VID_STRM_REMOVE,    /**改變流方向     * Change direction of a video stream. This operation can be used     * to activate or deactivate an existing video media. This will     * cause re-INVITE or UPDATE to be sent to remote party.     */    PJSUA_CALL_VID_STRM_CHANGE_DIR,    /**改變捕獲設備     * Change capture device of a video stream.  This will not send     * re-INVITE or UPDATE to remote party.     */    PJSUA_CALL_VID_STRM_CHANGE_CAP_DEV,    /**開啓流傳輸     * Start transmitting video stream. This will cause previously     * stopped stream to start transmitting again. Note that no     * re-INVITE/UPDATE is to be transmitted to remote since this     * operation only operates on local stream.     */    PJSUA_CALL_VID_STRM_START_TRANSMIT,    /**停止流傳輸     * Stop transmitting video stream. This will cause the stream to     * be paused in TX direction, causing it to stop sending any video     * packets. No re-INVITE/UPDATE is to be transmitted to remote     * with this operation.     */    PJSUA_CALL_VID_STRM_STOP_TRANSMIT,    /**傳輸關鍵幀     * Send keyframe in the video stream. This will force the stream to     * generate and send video keyframe as soon as possible. No     * re-INVITE/UPDATE is to be transmitted to remote with this operation.     */    PJSUA_CALL_VID_STRM_SEND_KEYFRAME} pjsua_call_vid_strm_op;

/** * Parameters for video stream operation on a call. */typedef struct pjsua_call_vid_strm_op_param{    /**     * Specify the media stream index. This can be set to -1 to denote     * the default video stream in the call, which is the first active     * video stream or any first video stream if none is active.     *     * This field is valid for all video stream operations, except     * PJSUA_CALL_VID_STRM_ADD.     *     * Default: -1 (first active video stream, or any first video stream     *              if none is active)     */    int med_idx;    /**     * Specify the media stream direction.     *     * This field is valid for the following video stream operations:     * PJSUA_CALL_VID_STRM_ADD and PJSUA_CALL_VID_STRM_CHANGE_DIR.     *     * Default: PJMEDIA_DIR_ENCODING_DECODING     */    pjmedia_dir dir;    /**     * Specify the video capture device ID. This can be set to     * PJMEDIA_VID_DEFAULT_CAPTURE_DEV to specify the default capture     * device as configured in the account.     *     * This field is valid for the following video stream operations:     * PJSUA_CALL_VID_STRM_ADD and PJSUA_CALL_VID_STRM_CHANGE_CAP_DEV.     *     * Default: capture device configured in account.     */    pjmedia_vid_dev_index cap_dev;} pjsua_call_vid_strm_op_param;


然後調用這個函數去配置：

/** * Add, remove, modify, and/or manipulate video media stream for the * specified call. This may trigger a re-INVITE or UPDATE to be sent * for the call. * * @param call_id    Call identification. * @param op        The video stream operation to be performed, *            possible values are #pjsua_call_vid_strm_op. * @param param        The parameters for the video stream operation, *            or NULL for the default parameter values *            (see #pjsua_call_vid_strm_op_param). * * @return        PJ_SUCCESS on success or the appropriate error. */PJ_DECL(pj_status_t) pjsua_call_set_vid_strm (                pjsua_call_id call_id,                pjsua_call_vid_strm_op op,                const pjsua_call_vid_strm_op_param *param);


檢索the call，獲得video stream index：

/** * Get the media stream index of the default video stream in the call. * Typically this will just retrieve the stream index of the first * activated video stream in the call. * * @param call_id    Call identification. * * @return        The media stream index or -1 if no video stream *             is present in the call. */PJ_DECL(int) pjsua_call_get_vid_stream_idx(pjsua_call_id call_id);


5.5 視頻呼叫信息：


pjsua_call_info 中增加了一些新的片段用來描述更多的媒體信息。

/** Number of media streams in this call */    unsigned        media_cnt;    /** Array of media stream information */    struct    {    /** Media index in SDP. */    unsigned        index;    /** Media type. */    pjmedia_type        type;    /** Media direction. */    pjmedia_dir        dir;    /** Call media status. */    pjsua_call_media_status status;    /** The specific media stream info. */    union {        /** Audio stream */        struct {        /** The conference port number for the call.  */        pjsua_conf_port_id   conf_slot;        } aud;        /** Video stream */        struct {        /**         * The window id for incoming video, if any, or         * PJSUA_INVALID_ID.         */        pjsua_vid_win_id     win_in;        /** The video capture device for outgoing transmission,         *  if any, or PJMEDIA_VID_INVALID_DEV         */        pjmedia_vid_dev_index    cap_dev;        } vid;    } stream;    } media[PJMEDIA_MAX_SDP_MEDIA];

還有以下幾類API，不一一列舉：


5.6 Video Call Stream Information and Statistic


5.7 Video Window API


5.8 Video Codec API