javascript - React - MediaSource video not rendering

Question

Hi we are working on group video chat, previously we were using webRTC however it didn't work for us. We are trying to achieve same via server instead. We are using MediaRecorder to capture our stream and pass it to server as base64 and sending chunks back to client.

However when we use MediaSource to append chunks to video element video is not render.

Any other ideas how we can achieve it?

Here is our code example.

  useEffect(() => {
    // Build new Hub Connection, url is currently hard coded.
    hubConnect = new HubConnectionBuilder()
      .withUrl(`/api/hubs/communication/test`, {
        accessTokenFactory: () => appState.jwtToken.replace("Bearer ", ""),
      })
      .build();

    hubConnect.start().then(() => {
      if (hubConnect) {
        var blobArray: any[] = [];

        let video: any = document.getElementById("video");

        if (!video) {
          return;
        }
        let mediaSource = new MediaSource();

        video.src = window.URL.createObjectURL(mediaSource);

        var sourceBuffer;

        const processBuffer = () => {
          try {
            if (mediaSource.readyState === "open" && sourceBuffer) {
              const blob = blobArray.shift();

              if (blob) {
                blob.arrayBuffer().then((a) => {
                  sourceBuffer.appendBuffer(a);
                });
              }
            }
          } catch (error) {}
        };

        mediaSource.addEventListener("sourceopen", () => {
          sourceBuffer = mediaSource.addSourceBuffer("video/webm; codecs=vp9");
        });

        mediaSource.addEventListener("updateend", () => {
          processBuffer();
        });

        hubConnect.on("newStream", (incomingBase64: string) => {
          blobArray.push(b64toBlob(incomingBase64));
          processBuffer();

   
          console.log(blobArray);
        });
      }
    });
  }, []);

  function b64toBlob(dataURI) {
    var byteString = atob(dataURI.split(",")[1]);
    var ab = new ArrayBuffer(byteString.length);
    var ia = new Uint8Array(ab);

    for (var i = 0; i < byteString.length; i++) {
      ia[i] = byteString.charCodeAt(i);
    }
    return new Blob([ab], { type: "video/webm; codecs=vp9" });
  }

Answer 1

等待大神答复