Android連接Kurento服務端通信過程

通信API

通信的過程是在封裝好的jar包裏面的，KurentoRoomAPI->(繼承)KurentoAPI->(實現)JsonRpcWebSocketClient.WebSocketConnectionEvents。

public interface WebSocketConnectionEvents {

public void onOpen(ServerHandshake handshakedata);

public void onRequest(JsonRpcRequest request);

public void onResponse(JsonRpcResponse response);

public void onNotification(JsonRpcNotification notification);

public void onClose(int code, String reason, boolean remote);

public void onError(Exception e);

}

連接Url

KurentoAPI.connectWebSocket

    public void connectWebSocket() {
        try {
            if(isWebSocketConnected()){
                return;
            }
            URI uri = new URI(wsUri);
            client = new JsonRpcWebSocketClient(uri, this,executor);
            if (webSocketClientFactory != null) {
                client.setWebSocketFactory(webSocketClientFactory);
            }
            executor.execute(new Runnable() {
                public void run() {
                    client.connect();
                }
            });
        } catch (Exception exc){
            Log.e(LOG_TAG, "connectWebSocket", exc);
        }
    }

2)發送消息

KurentoAPI.send

    protected void send(String method, HashMap<String, Object> namedParameters, int id){

        try {
            final JsonRpcRequest request = new JsonRpcRequest();
            request.setMethod(method);
            if(namedParameters!=null) {
                request.setNamedParams(namedParameters);
            }
            if(id>=0) {
                request.setId(id);
            }
            executor.execute(new Runnable() {
                public void run() {
                    if(isWebSocketConnected()) {
                        client.sendRequest(request);
                    }
                }
            });
        } catch (Exception exc){
            Log.e(LOG_TAG, "send: "+method, exc);
        }
    }

KurentoRoomAPI中實現了具體發送業務消息的代碼

如加入聊天室：

    public void sendJoinRoom(String userId, String roomId, boolean dataChannelsEnabled, int id){
        HashMap<String, Object> namedParameters = new HashMap<>();
        namedParameters.put("user", userId);
        namedParameters.put("room", roomId);
        namedParameters.put("dataChannels", dataChannelsEnabled);
        send("joinRoom", namedParameters, id);
    }

而在項目中自己寫的類VideoClientAPI又繼承自KurentoRoomAPI並實現獲取WebRTC監控視頻的功能。

服務端還是原來的基於原本的js-kurento-player的服務端，把需要的攝像頭的rtsp地址用videourl傳給服務端，接入原本的webrtc的過程。

VideoClientAPI.startPlay

   public void startPlay(String videourl, String sdpOffer, int id) {
        HashMap<String, Object> namedParameters = new HashMap<>();
        namedParameters.put("videourl", videourl);
        namedParameters.put("sdpOffer", sdpOffer);
        sendWithBuffer("start", namedParameters, id);
    }

具體的webrtc的過程暫時不討論。

另外一個自定義的消息是獲取視頻圖片

   public void getImage(int id) {
        HashMap<String, Object> namedParameters = new HashMap<>();
        sendWithBuffer("getImage", namedParameters, id);
    }

3)接受消息

javascript的websocket建立連接後獲取到服務端消息後會進入到onMessage事件方法中，在裏面根據不同的消息做不同的處理。

javascript裏面的websocket對象，對應於前面KurentoAPI.client對象（JsonRpcWebSocketClient類）

相對應的也有onMessage

@Override
		public void onMessage(final String message) {
			executor.execute(new Runnable() {
				@Override
				public void run() {
					if (connectionState == WebSocketConnectionState.CONNECTED) {
						try {
							JSONRPC2Message msg = JSONRPC2Message.parse(message);

							if (msg instanceof JSONRPC2Request) {
								JsonRpcRequest request = new JsonRpcRequest();
								request.setId(((JSONRPC2Request) msg).getID());
								request.setMethod(((JSONRPC2Request) msg).getMethod());
								request.setNamedParams(((JSONRPC2Request) msg).getNamedParams());
								request.setPositionalParams(((JSONRPC2Request) msg).getPositionalParams());
								events.onRequest(request);
							} else if (msg instanceof JSONRPC2Notification) {
								JsonRpcNotification notification = new JsonRpcNotification();
								notification.setMethod(((JSONRPC2Notification) msg).getMethod());
								notification.setNamedParams(((JSONRPC2Notification) msg).getNamedParams());
								notification.setPositionalParams(((JSONRPC2Notification) msg).getPositionalParams());
								events.onNotification(notification);
							} else if (msg instanceof JSONRPC2Response) {
								JsonRpcResponse notification = new JsonRpcResponse(message);
								events.onResponse(notification);
							}
						} catch (JSONRPC2ParseException e) {
							// TODO: Handle exception
						}
					}
				}
			});
		}

這裏接收到一個json字符串並解析，根據json解析後的類型不同，後續調用onRequest，onResponse或者onNotification，就是JsonRpcWebSocketClient.WebSocketConnectionEvents接口中定義的那幾個方法，在KurentoRoomAPI中實現了具體的代碼。

@Override
    public void onResponse(JsonRpcResponse response) {
        if(response.isSuccessful()){
            JSONObject jsonObject = (JSONObject)response.getResult();
            RoomResponse roomResponse = new RoomResponse(response.getId().toString(), jsonObject);

            synchronized (listeners) {
                for (RoomListener rl : listeners) {
                    rl.onRoomResponse(roomResponse);
                }
            }
        } else {
            RoomError roomError = new RoomError(response.getError());

            synchronized (listeners) {
                for (RoomListener rl : listeners) {
                    rl.onRoomError(roomError);
                }
            }
        }
    }

    /**
     * Callback method that relays the RoomNotification to the RoomListener interface.
     */
    @Override
    public void onNotification(JsonRpcNotification notification) {
        RoomNotification roomNotification = new RoomNotification(notification);

        synchronized (listeners) {
            for (RoomListener rl : listeners) {
                rl.onRoomNotification(roomNotification);
            }
        }
    }

這裏onResponse封裝了一下，把信息保存到RoomResponse裏面，並通過RoomListener接口的onRoomResponse將消息發給前端代碼。

onNotifaction也封裝了一下，保持到RoomNotifacation裏面，並通過RoomListener接口的onRoomNotification將消息發給前端代碼。

前端業務邏輯代碼需要實現RoomListener。

public interface RoomListener {

    public void onRoomResponse(RoomResponse response);

    public void onRoomError(RoomError error);

    public void onRoomNotification(RoomNotification notification);

    public void onRoomConnected();

    public void onRoomDisconnected();
}

具體前端代碼：

public class VideoPlayer implements NBMWebRTCPeer.Observer, RoomListener {

具體的接收消息並處理的代碼就在VideoPlayer的onRoomNotifacation裏面了。

@Override
    public void onRoomNotification(RoomNotification notification) {
        //MyLog.i("RoomListener", "onRoomNotification:" + notification);
        String method = notification.getMethod();
        switch (method) {
            case "startResponse":
                startResponse(notification);
                break;
            case "error":
//                if (state == I_AM_STARTING) {
//                    setState(I_CAN_START);
//                }
//                onError('Error message from server: ' + parsedMessage.message);
                break;
            case "playEnd":
                //playEnd();
                break;
            case "videoInfo":
                //showVideoData(parsedMessage);
                break;
            case "iceCandidate":
                addIceCandidate(notification);
                break;
            case "seek":
                //console.log (parsedMessage.message);
                break;
            case "position":
                //document.getElementById("videoPosition").value = parsedMessage.position;
                break;
            case "imageInfo":
                getImageInfo(notification);
                break;
            default:
//                if (state == I_AM_STARTING) {
//                    setState(I_CAN_START);
//                }
//                onError('Unrecognized message', parsedMessage);
                break;
        }
    }

VideoPlayer也是封裝了視頻通信的一個類，具體到界面相關的交互還要到Activity裏面。

public class VideoClientActivity
        extends AppCompatActivity
{

    protected VideoPlayer videoPlayer=new VideoPlayer(this);

在Activiy裏面調用videoPlayer裏面的方法，發送消息，如：

        findViewById(R.id.btnGenerate).setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                VideoDebugActivity activity = VideoDebugActivity.this;
                videoPlayer.startPlay(cbSTUN.isChecked(), cbUseBuffer.isChecked(), true, cbLocalMedia.isChecked());
            }
        });

處理消息：

videoPlayer.addRtcListener(new RtcListener() {
            @Override
            public void addIceCandidate(RoomNotification notification) {
                if (isShowLocalCandidate == false) {
                    showRemoteCandidates();
                }
            }

            @Override
            public void onIceCandidate(IceCandidate iceCandidate, NBMPeerConnection connection) {
                if (isShowLocalCandidate)
                    showLocalCandidates();
            }

            @Override
            public void onIceGatheringDone() {
                Logger.w("OnIceGatheringDone");
                //todo:顯示offer和localCandidate
            }

            @Override
            public void getImageInfo(String base64) {
                //final Bitmap bitmap = base64ToBitmap(base64);
                //todo:顯示圖片
//                runOnUiThread(new Runnable() {
//                    @Override
//                    public void run() {
//                        videoImage.setImageBitmap(bitmap);
//                    }
//                });

            }
        });

這裏向videoPlayer添加RtcListener，從剛剛的onRoomNotification將消息觸發到Activity進行相應處理。

這裏用到java的接口的匿名實現的概念，對應於c#的事件(+=相應函數)。

這裏的RtcListener是自定義的接口，用於做事件響應的過程。

public interface RtcListener  {
    void addIceCandidate(RoomNotification notification) ;
    void onIceCandidate(IceCandidate iceCandidate, NBMPeerConnection connection);
    void onIceGatheringDone();
    void getImageInfo(String base64) ;
}

需要相應其他的服務端消息時，這樣也要相應的增加接口。

具體調用路徑：VideoPlayer的onRoomNotification->getImageInfo->notifyGetImageInfo->Activity中定位的RtcListener匿名實現類

Android連接Kurento服務端通信過程

SQL優化-20231016

3dmax導入unity問題(1) 軸角度座標

ShaderGraph學習(HDRP)

UnityWebGL調研(6) SignalR通信

MAXScript學習筆記(3) 功能:分離元素

視頻服務器(6) Kurento[1] rtsp2webrtc

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結