javascript - Is there any way to change the gain individually in real time when playing a node that is a composite of two AudioBuffers?

Question

I was having the following problems.
When I run AudioBufferSourceNode.start() when I have multiple tracks, I sometimes get a delay
Then, per chrisguttandin's answer, I tried the method using offLineAudioContext. (Thanks to chrisguttandin).

---

I wanted to play two different mp3 files completely simultaneously, so I used offlineAudioContext to synthesize an audioBuffer.
And I succeeded in playing the synthesized node. The following is a demo of it.
CodeSandBox
The code in the demo is based on the code in the following page.
OfflineAudioContext - Web APIs | MDN

However, the demo does not allow you to change the gain for each of the two types of audio.
Is there any way to change the gain of the two types of audio during playback?

What I would like to do is as follows.

• I want to play two pieces of audio perfectly simultaneously.
• I want to change the gain of each of the two audios in real time.

Therefore, if you can achieve what you want to do as described above, you don't need to use offlineAudioContext.

The only way I can think of to do this is to run startRendering on every input type="range", but I don't think this is practical from a performance standpoint.
Also, I looked for a solution to this problem, but could not find one.

code

let ctx = new AudioContext(),
  offlineCtx,
  tr1,
  tr2,
  renderedBuffer,
  renderedTrack,
  tr1gain,
  tr2gain,
  start = false;

const trackArray = ["track1", "track2"];

const App = () => {
  const [loading, setLoading] = useState(true);
  useEffect(() => {
    (async () => {
      const bufferArray = trackArray.map(async (track) => {
        const res = await fetch("/" + track + ".mp3");
        const arrayBuffer = await res.arrayBuffer();
        return await ctx.decodeAudioData(arrayBuffer);
      });
      const audioBufferArray = await Promise.all(bufferArray);
      const source = audioBufferArray[0];
      offlineCtx = new OfflineAudioContext(
        source.numberOfChannels,
        source.length,
        source.sampleRate
      );
      tr1 = offlineCtx.createBufferSource();
      tr2 = offlineCtx.createBufferSource();
      tr1gain = offlineCtx.createGain();
      tr2gain = offlineCtx.createGain();
      tr1.buffer = audioBufferArray[0];
      tr2.buffer = audioBufferArray[1];
      tr1.connect(tr1gain);
      tr1gain.connect(offlineCtx.destination);
      tr2.connect(tr1gain);
      tr2gain.connect(offlineCtx.destination);
      tr1.start();
      tr2.start();
      offlineCtx.startRendering().then((buffer) => {
        renderedBuffer = buffer;
        renderedTrack = ctx.createBufferSource();
        renderedTrack.buffer = renderedBuffer;
        setLoading(false);
      });
    })();

    return () => {
      ctx.close();
    };
  }, []);

  const [playing, setPlaying] = useState(false);
  const playAudio = () => {
    if (!start) {
      renderedTrack = ctx.createBufferSource();
      renderedTrack.buffer = renderedBuffer;
      renderedTrack.connect(ctx.destination);
      renderedTrack.start();
      setPlaying(true);
      start = true;
      return;
    }
    ctx.resume();
    setPlaying(true);
  };
  const pauseAudio = () => {
    ctx.suspend();
    setPlaying(false);
  };
  const stopAudio = () => {
    renderedTrack.disconnect();
    start = false;
    setPlaying(false);
  };

  const changeVolume = (e) => {
    const target = e.target.ariaLabel;
    target === "track1"
      ? (tr1gain.gain.value = e.target.value)
      : (tr2gain.gain.value = e.target.value);
  };
  const Inputs = trackArray.map((track, index) => (
    <div key={index}>
      <span>{track}</span>
      <input
        type="range"
        onChange={changeVolume}
        step="any"
        max="1"
        aria-label={track}
        disabled={loading ? true : false}
      />
    </div>
  ));

  return (
    <>
      <button
        onClick={playing ? pauseAudio : playAudio}
        disabled={loading ? true : false}
      >
        {playing ? "pause" : "play"}
      </button>
      <button onClick={stopAudio} disabled={loading ? true : false}>
        stop
      </button>
      {Inputs}
    </>
  );
};

question from:https://stackoverflow.com/questions/65919632/is-there-any-way-to-change-the-gain-individually-in-real-time-when-playing-a-nod

Answer 1

As a test, I'd go back to your original solution, but instead of

tr1.start();
tr2.start();

try something like

t = ctx.currentTime;
tr1.start(t+0.1);
tr2.start(t+0.1);

There will be a delay of about 100 ms before audio starts, but they should be synchronized precisely. If this works, reduce the 0.1 to something smaller, but not zero. Once this is working, you can then connect separate gain nodes to each track and control the gains of each in real-time.

Oh, one other thing, instead of resuming the context after calling start, you might want to do something like

ctx.resume()
  .then(() => {
    let t = ctx.currentTime;
    tr1.start(t + 0.1);
    tr2.start(t + 0.1);
  });

The clock isn't running if the context is suspended, and resuming doesn't happen instantly. It may take some time to restart the audio HW.