iPhone/iPad札記2012/03/11 - 1 音頻，混音

1. conference音頻問題總結

＊ 音頻雙向都通過，但不穩定，時好時壞（錄音只成功過一次，播放曾有過連續工作20－30分鐘一切正常的記錄，但重啓系統後就出現問題）

    Q Action: to see how often the callback would be called ?

        A: playing: 10 ms, recording 3ms 

＊ 是否應該使用Audio Unit而非Audio Queue來實現VOIP功能？

     ＃ 據資料說，Audio Queue Service的callback訪問時間在0.5秒以上

     ＃ Audio Unit資料說它比較適用於需要low-lantency的音頻應用，如VOIP( used high-priority thread)

      REFERENCE: iOS provides audio processing plug-ins that support mixing, equalization, format conversion, and realtime input/output for recording, playback, offline rendering, and live conversation such as for VoIP (Voice over Internet Protocol). 

* 一個新的思路：

      # 另起一個thread去讀session層的audio data，並把它們放進一個queue

      # Audio Queue callback從新的queue裏拿數據去放

＊實驗：

      ＃爲什麼iOS主線程readAudioData會有問題，而單獨的thread則沒有問題？

      ＃ 在windows/Android環境裏重現相關speex的問題

＊Audio Engine

      ＃審閱相關代碼

 ＊明天建立一個內網上的測試環境，杜絕公網上的網絡狀況的影響

  * 還可以做的嘗試：把Speex換成G711

 

2. MixerHost sample code 

* configureAndInitializeAudioProcessingGraph

registration as follows 

        // Setup the struture that contains the input render callback 

        AURenderCallbackStruct inputCallbackStruct;

        inputCallbackStruct.inputProc        = &inputRenderCallback;

        inputCallbackStruct.inputProcRefCon  = soundStructArray;

  ^ this is an array of sound data 

        // Set a callback for the specified node's specified input

        result = AUGraphSetNodeInputCallback (

                     processingGraph,

                     mixerNode,

                     busNumber,

                     &inputCallbackStruct

                 );

sound struct 

typedef struct {

    BOOL                 isStereo;           // set to true if there is data in the audioDataRight member

    UInt32               frameCount;         // the total number of frames in the audio data

    UInt32               sampleNumber;       // the next audio sample to play

    AudioUnitSampleType  *audioDataLeft;     // the complete left (or mono) channel of audio data read from an audio file

    AudioUnitSampleType  *audioDataRight;    // the complete right channel of audio data read from an audio file

    

} soundStruct, *soundStructPtr;

3. Reading "Audio Unit " related guides - Component Service Reference 

* AURenderCallback 

typedef OSStatus (*AURenderCallback) (
   void                        *inRefCon,
   AudioUnitRenderActionFlags  *ioActionFlags,
   const AudioTimeStamp        *inTimeStamp,
   UInt32                      inBusNumber,
   UInt32                      inNumberFrames,
   AudioBufferList             *ioData
);

inRefCon

Custom data that you provided when registering your callback with the audio unit.

4. Test555程序

＊ 試驗callback被回調的頻率

    Q Action: to see how often the callback would be called ?

        A: playing: 10 ms, recording 3ms 

    Result: 

notification: Invalid mode encountered. The stream is corrupted.

2012-03-11 10:52:10.209 test555[324:207] AQBufferCallback...

2012-03-11 10:52:10.219 test555[324:207] AQBufferCallback...

2012-03-11 10:52:10.229 test555[324:207] AQBufferCallback...

2012-03-11 10:52:10.239 test555[324:207] AQBufferCallback...

[Session started at 2012-03-11 10:58:34 +0800.]

2012-03-11 10:58:34.707 test555[368:7303] InputBufferHandler...

2012-03-11 10:58:34.710 test555[368:7303] audio data sent to audio engine...

2012-03-11 10:58:34.710 test555[368:7303] InputBufferHandler...

2012-03-11 10:58:34.712 test555[368:7303] audio data sent to audio engine...

2012-03-11 10:58:34.713 test555[368:7303] InputBufferHandler...

2012-03-11 10:58:34.714 test555[368:7303] audio data sent to audio engine...

2012-03-11 10:58:34.715 test555[368:7303] InputBufferHandler...

2012-03-11 10:58:34.717 test555[368:7303] audio data sent to audio engine...