上一篇闡述了調研結果,而我們常用的應用場景就是錄制屏幕內容,然后將內容分享給他人(直播或錄播)。流程如下:
1.被錄制端host app需引入 ReplayKit,以便可以使用其api選擇一個app的extension來啟動錄制;
2.廣播端宿主app需要集成 Broadcast UI 和 Broadcast Upload 兩個 Extension,以便出現在被錄制端可選的 App 列表中;
3.host app選定宿主app 后,將啟動宿主app的extension,開始錄制和廣播相關邏輯。
上文已經提到,從iOS9系統開始,蘋果推出了replaykit 這個sdk來支持屏幕錄制,通過extension形式實現屏幕錄制。本文將對屏幕錄制使用replaykit的技術細節進行描述, 下一篇將對錄制內容的推送(廣播)進行描述。通過本文你將對以下幾方面得到信息:
1. extension是什么?
2. extension跟app什么關系?
3. 在iOS10 11上集成extension注意哪些,區別有哪些?
4. 調試時注意哪些?
5. 調試時涉及到的原理和通信方式
extension是什么?
- 邏輯形式:
extension必須寄生在宿主app中,會隨著宿主 app的安裝而安裝,同時隨著宿主 app的卸載而卸載,但是extension卻可以獨立生存,即使宿主app沒有啟動,extension也可以為其他app提供相關服務。(能夠調起extension的app被稱為host app) - 物理形式:
iOS系統提供屏幕錄制和直播功能都需要通過Extensions的形式來支持,通過在Xcode的已有工程中新建target,選擇broadcast upload extension,這樣工程中將自動添加broadcast upload extension和broadcast setup UI extension兩個extensions。extension并不是一個獨立的app,它有一個包含在app bundle中的獨立bundle,extension的bundle后綴名是.appex。
集成extension
集成方式很簡單,新建target,選擇upload相關兩個extension。集成之后將在工程的列表中看到兩個新增的目錄。
需要注意的是,ios10 系統在upload的extension中的info.plist中NSExtensionPointIdentifier對應的value必須使用NSExtensionPointIdentifierkey對應ios10才兼容的com.apple.broadcast-services,不應該使用com.apple.broadcast-services-upload ,在iOS10系統中使用com.apple.broadcast-services-upload將無法通過編譯,Xcode會報錯。
通信
iOS10系統和iOS11系統的屏幕錄制和直播,涉及到extensions和host app、containing app之間的通信,其中host app一端需要集成ReplayKit2,從而可以發起錄制和直播請求,而containing app需要集成extensions,實現對其他可以錄制的app的直播功能的支持。extension和host app之間可以通過extensionContext屬性直接通信,extension和宿主containing app之間是通過IPC或基于group的文件共享來實現的。
對于iOS10和iOS11,屏幕錄制區別較大,前者只能錄制app內的內容,后者可以錄制整個系統的內容,而且前者可以通過代碼控制錄制的啟動,而后者只能通過用戶的操作(控制中心,點擊圓點,選擇app)啟動錄制。
iOS 10
在iOS10系統中,想要錄制當前app內的內容,必須通過其他app的extension,而啟動這個extension必須通過集成replaykit的api。
@interface RPBroadcastActivityViewController : UIViewController
+ (void)loadBroadcastActivityViewControllerWithHandler:(void(^)(RPBroadcastActivityViewController * _Nullable broadcastActivityViewController, NSError * _Nullable error))handler;
+ (void)loadBroadcastActivityViewControllerWithPreferredExtension:(NSString * _Nullable)preferredExtension handler:(nonnull void(^)(RPBroadcastActivityViewController * _Nullable broadcastActivityViewController, NSError * _Nullable error))handler API_AVAILABLE(ios(11.0)) API_UNAVAILABLE(tvos);
@end
@protocol RPBroadcastActivityViewControllerDelegate <NSObject>
- (void)broadcastActivityViewController:(RPBroadcastActivityViewController *)broadcastActivityViewController didFinishWithBroadcastController:(nullable RPBroadcastController *)broadcastController error:(nullable NSError *)error API_AVAILABLE(ios(10.0), tvos(10.0));
@end
按照前文流程,當host app一端想要將app或系統內容廣播給他人觀看時,需要首先選擇一個app的extension來幫他廣播,就是需要展示出支持廣播的app列表。這點通過調用ReplayKit2的RPBroadcastActivityViewController類的load相關api來實現。可以看到上面有兩個api可供使用。
-
(void)loadBroadcastActivityViewControllerWithHandler:(void(^)(RPBroadcastActivityViewController * _Nullable broadcastActivityViewController, NSError * _Nullable error))handler;
![
這時iOS系統將會尋找系統內已經集成了屏幕錄制和直播extensions的containing app,并將這些app列表展示出來,用戶可以在列表中選擇containing app,點擊選擇之后,將通過containing app的extension中的UI-extension來展示相關的界面(可以自定義),讓用戶輸入信息,一般用來鑒權或者保存用戶信息,用戶點擊ok按鈕之后,可以通過相關方法來調用[self.extensionContext completeRequestWithBroadcastURL:broadcastURL broadcastConfiguration:broadcastConfig setupInfo:setupInfo];,這個方法中將傳遞一些信息給host app,RPBroadcastActivityViewControllerDelegate的代理方法didFinishWithBroadcastController將會回調調用,這時我們可以獲取到用于廣播的controller,相當于與containing app已經建立起了通信鏈路,然后調用broadcastController 的startBroadcastWithHandler接口即可啟動錄制。
image.png -
(void)loadBroadcastActivityViewControllerWithPreferredExtension:(NSString * _Nullable)preferredExtension handler:(nonnull void(^)(RPBroadcastActivityViewController * _Nullable broadcastActivityViewController, NSError * _Nullable error))handler
第二個api是ios11新增的。可以通過參數preferredExtension,直接打開指定使用的app,只需要preferredExtension傳遞相應app extension的bundle id。RPBroadcastActivityViewControllerDelegate的代理方法didFinishWithBroadcastController將會回調調用,這時我們可以獲取到用于廣播的controller,相當于與containing app已經建立起了通信鏈路,然后調用broadcastController 的startBroadcastWithHandler接口即可啟動錄制。
image.png
iOS11
在iOS10系統中,只能用戶自己手動啟動錄制,并且無法通過代碼控制錄制進程的啟動,所以被錄制端host app其實無需集成replaykit,而只需要宿主app集成兩個extension。
與iOS10不同的是,用戶手動選擇錄制app后,宿主app的extension相關方法將自動開始回調。
錄制進程
通過上面的形式,啟動錄制后,我們可以在extension中自建出來的SampleHandler文件中相關代理方法中獲取到屏幕采集的進度,具體使用方式見注釋:
@interface RPBroadcastSampleHandler : RPBroadcastHandler
/*! @abstract Method is called when the RPBroadcastController startBroadcast method is called from the broadcasting application.
@param setupInfo Dictionary that can be supplied by the UI extension to the sample handler.
屏幕采集工作已經開始啟動,在此方法中一般進行初始化工作
*/
- (void)broadcastStartedWithSetupInfo:(nullable NSDictionary <NSString *, NSObject *> *)setupInfo;
/*! @abstract Method is called when the RPBroadcastController pauseBroadcast method is called from the broadcasting application. */
- (void)broadcastPaused;
/*! @abstract Method is called when the RPBroadcastController resumeBroadcast method is called from the broadcasting application. */
- (void)broadcastResumed;
/*! @abstract Method is called when the RPBroadcastController finishBroadcast method is called from the broadcasting application. */
- (void)broadcastFinished;
/*! @abstract Method is called when broadcast is started from Control Center and provides extension information about the first application opened or used during the broadcast.
@param applicationInfo Dictionary that contains information about the first application opened or used buring the broadcast.
*/
- (void)broadcastAnnotatedWithApplicationInfo:(NSDictionary *)applicationInfo API_AVAILABLE(ios(11.2)) API_UNAVAILABLE(tvos);
/*! @abstract Method is called as video and audio data become available during a broadcast session and is delivered as CMSampleBuffer objects.
@param sampleBuffer CMSampleBuffer object which contains either video or audio data.
@param sampleBufferType Determine's the type of the sample buffer defined by the RPSampleBufferType enum.
采集到數據的實時回調,此方法中的sampleBuffer數據結構中有視頻和音頻數據,我們通過相關推流方法將數據推送給服務器,即實現了錄制和推流。
*/
- (void)processSampleBuffer:(CMSampleBufferRef)sampleBuffer withType:(RPSampleBufferType)sampleBufferType;
/*! @abstract Method that should be called when broadcasting can not proceed due to an error. Calling this method will stop the broadcast and deliver the error back to the broadcasting app through RPBroadcastController's delegate.
@param error NSError object that will be passed back to the broadcasting app through RPBroadcastControllerDelegate's broadcastController:didFinishWithError: method.
*/
- (void)finishBroadcastWithError:(NSError *)error;
@end
文件讀寫
盡管extension的bundle是放在containing app的bundle中,但是他們是兩個完全獨立的進程,之間不能直接通信。不過extension可以通過openURL的方式啟動containing app(當然也能啟動其它app),不過extension中是無法直接使用openURL的,必須通過extensionContext借助host app來實現。extension和containing app可以共同讀寫一個被稱為Shared resources的存儲區域,這是通過App Groups實現的,用于同一group下的app共享同一份讀寫空間,以實現數據共享。
? 首先需要在apple開發網站上對profile文件進行配置,將group數據共享配置,并設置group id(dns域名反寫),用戶app和extension之間;
? 然后app中配置這個profile,并設置app的group,通過TARGETS-->App-->Capabilities-->App Groups,選擇正確的group id;
? 同時,在extension中也要通過TARGETS-->App-->Capabilities-->App Groups,選擇同樣的group id;
? 通過NSUserDefaults共享數據,通過下面的形式:
- (void)saveTextByNSUserDefaults
{
NSUserDefaults *shared = [[NSUserDefaults alloc] initWithSuiteName:@"group.cmcc.ShareScreen"];
[shared setObject:_textField.text forKey:@"cmcc"];
[shared synchronize];
}
? 讀寫文件時,也需要通過指定group id的形式,才能將文件寫入共享的數據區,或者從共享數據區讀出來
- (NSString *)readTextByNSFileManager
{
NSError *err = nil;
NSURL *containerURL = [[NSFileManager defaultManager] containerURLForSecurityApplicationGroupIdentifier:@"group.cmcc.ShareScreen "];
containerURL = [containerURL URLByAppendingPathComponent:@"Library/Caches/good"];
NSString *value = [NSString stringWithContentsOfURL:containerURL encoding:NSUTF8StringEncoding error:&err];
return value;
}
- (bool)writeTextByNSFileManager
{
NSError *err = nil;
NSURL *containerURL = [[NSFileManager defaultManager] containerURLForSecurityApplicationGroupIdentifier:@"group.cmcc.ShareScreen "];
containerURL = [containerURL URLByAppendingPathComponent:@"Library/Caches/good"];
NSString *value = @"just test";
BOOL result = [value writeToURL:containerURL atomically:YES encoding:NSUTF8StringEncoding error:&err];
return result;
}
注意:containing app需要配置帶有group配置的profile, extension可以配置自動,但是bundle id不能和containing app相同
調試
-
由于涉及到extensions作為獨立target,所以調試時,需要單獨編譯運行,即我們想要調試containing app那就需要將xcode切換到containing app,然后重新運行,如果需要調試upload 或 setupUI的extension,那就需要需要切換到extension的target,在重新運行,這樣才能在sampleHandler相關的方法中斷點調試;
image.png userDidFinishSetup(通過extensionContext與host app通信的方法)必須在viewDidAppear后,而不能放在viewDidLoad之后,否則導致無法將事件傳遞給SampleHandler,它的代理方法不會回調。
- (void)viewDidAppear:(BOOL)animated
{
[super viewDidAppear:animated];
[self userDidFinishSetup];
}