人臉識別技術 (二) —— 基于CoreImage實現(xiàn)視頻中人臉的識別

版本記錄

版本號 時間
V1.0 2018.01.31

前言

人臉識別是圖像識別技術中的一種,廣泛的應用于很多領域,接下來這幾篇我們就一起來研究幾種關于人臉識別的技術。感興趣的可以參考上面幾篇文章。
1. 人臉識別技術 (一) —— 基于CoreImage實現(xiàn)對靜止圖片中人臉的識別

基于CoreImage的視頻中人臉識別技術

第一篇文章我們利用CoreImage對靜止的圖像進行人臉識別,相對來說,靜止圖像還是好識別的,如果要識別由攝像頭采集來的視頻中的人臉,那就相對來說難了,因為會有很多的性能問題。下面我們就一起看一下,利用AVFoundation進行圖像采集,利用CoreImage識別視頻中的人臉。


功能實現(xiàn)

還是直接看一下代碼。

#import "ViewController.h"
#import <AVFoundation/AVFoundation.h>

@interface ViewController () <AVCaptureVideoDataOutputSampleBufferDelegate>

@property (nonatomic, strong) AVCaptureSession *captureSession;
@property (nonatomic, strong) AVCaptureDevice *captureDevice;
@property (nonatomic, strong) AVCaptureDeviceInput *captureVideoDeviceInput;
@property (nonatomic, strong) AVCaptureVideoDataOutput *captureMovieFileOutput;
@property (nonatomic, strong) AVCaptureConnection *captureConnection;
@property (nonatomic, strong) AVCaptureVideoPreviewLayer *previewLayer;
@property (nonatomic, strong) NSMutableArray <UIView *> *faceViewArrM;

@end

@implementation ViewController

- (void)viewDidLoad
{
    [super viewDidLoad];
    
    self.faceViewArrM = [NSMutableArray array];
    
    self.captureSession = [[AVCaptureSession alloc] init];
    if ([self.captureSession canSetSessionPreset:AVCaptureSessionPresetHigh]) {
        self.captureSession.sessionPreset = AVCaptureSessionPresetHigh;
    }
    else {
        self.captureSession.sessionPreset = AVCaptureSessionPreset1280x720;
    }
    
    for (AVCaptureDevice *device in [AVCaptureDevice devices]) {
        if ([device hasMediaType:AVMediaTypeVideo]) {
            if (device.position == AVCaptureDevicePositionFront) {
                self.captureDevice = device;
            }
        }
    }
    
    //添加輸入
    [self addVideoInput];
    
    //添加輸出
    [self addVideoOutput];
    
    //添加預覽圖層
    [self addPreviewLayer];
    
    [self.captureSession commitConfiguration];
    [self.captureSession startRunning];

}

#pragma mark -  Object Private Function

- (void)addVideoInput
{
    NSError *error;
    self.captureVideoDeviceInput = [AVCaptureDeviceInput deviceInputWithDevice:self.captureDevice error:&error];
    if (error) {
        return;
    }
    if ([self.captureSession canAddInput:self.captureVideoDeviceInput]) {
        [self.captureSession addInput:self.captureVideoDeviceInput];
    }
}

- (void)addVideoOutput
{
    self.captureMovieFileOutput = [[AVCaptureVideoDataOutput alloc] init];
    [self.captureMovieFileOutput setSampleBufferDelegate:self queue:dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0)];
    self.captureMovieFileOutput.videoSettings = [NSDictionary dictionaryWithObjectsAndKeys:[NSNumber numberWithUnsignedInt:kCVPixelFormatType_32BGRA], (id)kCVPixelBufferPixelFormatTypeKey, nil];
    if ([self.captureSession canAddOutput:self.captureMovieFileOutput]) {
        [self.captureSession addOutput:self.captureMovieFileOutput];
    }
    
    //設置鏈接管理對象
    self.captureConnection = [self.captureMovieFileOutput connectionWithMediaType:AVMediaTypeVideo];
    //視頻旋轉方向設置
    self.captureConnection.videoScaleAndCropFactor = self.captureConnection.videoMaxScaleAndCropFactor;;
    //視頻穩(wěn)定設置
    if ([self.captureConnection isVideoStabilizationSupported]) {
        self.captureConnection.preferredVideoStabilizationMode = AVCaptureVideoStabilizationModeAuto;
    }
    
//    AVCaptureFileOutputDelegate *del = nil;
}

- (void)addPreviewLayer
{
    self.previewLayer = [AVCaptureVideoPreviewLayer layerWithSession:self.captureSession];
    [self.previewLayer setVideoGravity:AVLayerVideoGravityResizeAspect];
    self.previewLayer.frame = self.view.bounds;
    [self.view.layer addSublayer:self.previewLayer];
}

- (void)detectFaceWithImage:(UIImage *)image
{
    // 圖像識別能力:可以在CIDetectorAccuracyHigh(較強的處理能力)與CIDetectorAccuracyLow(較弱的處理能力)中選擇,因為想讓準確度高一些在這里選擇CIDetectorAccuracyHigh
    NSDictionary *opts = [NSDictionary dictionaryWithObject:
                          CIDetectorAccuracyHigh forKey:CIDetectorAccuracy];
    // 將圖像轉換為CIImage
    CIImage *faceImage = [CIImage imageWithCGImage:image.CGImage];
    CIDetector *faceDetector = [CIDetector detectorOfType:CIDetectorTypeFace context:nil options:opts];
    // 識別出人臉數(shù)組
    NSArray *features = [faceDetector featuresInImage:faceImage];
    // 得到圖片的尺寸
    CGSize inputImageSize = [faceImage extent].size;
    //將image沿y軸對稱
    CGAffineTransform transform = CGAffineTransformScale(CGAffineTransformIdentity, 1, -1);
    //將圖片上移
    transform = CGAffineTransformTranslate(transform, 0, -inputImageSize.height);
    
    //清空數(shù)組
    dispatch_async(dispatch_get_main_queue(), ^{
        [self.faceViewArrM enumerateObjectsUsingBlock:^(UIView * _Nonnull obj, NSUInteger idx, BOOL * _Nonnull stop) {
            [obj removeFromSuperview];
             obj = nil;
        }];
    });
    
    // 取出所有人臉
    for (CIFaceFeature *faceFeature in features){
        //獲取人臉的frame
        CGRect faceViewBounds = CGRectApplyAffineTransform(faceFeature.bounds, transform);
        CGSize viewSize = self.previewLayer.bounds.size;
        CGFloat scale = MIN(viewSize.width / inputImageSize.width,
                            viewSize.height / inputImageSize.height);
        CGFloat offsetX = (viewSize.width - inputImageSize.width * scale) / 2;
        CGFloat offsetY = (viewSize.height - inputImageSize.height * scale) / 2;
        // 縮放
        CGAffineTransform scaleTransform = CGAffineTransformMakeScale(scale, scale);
        // 修正
        faceViewBounds = CGRectApplyAffineTransform(faceViewBounds,scaleTransform);
        faceViewBounds.origin.x += offsetX;
        faceViewBounds.origin.y += offsetY;
        
        //描繪人臉區(qū)域
        dispatch_async(dispatch_get_main_queue(), ^{
            UIView* faceView = [[UIView alloc] initWithFrame:faceViewBounds];
            faceView.layer.borderWidth = 2;
            faceView.layer.borderColor = [[UIColor redColor] CGColor];
            [self.view addSubview:faceView];
            [self.faceViewArrM addObject:faceView];
        });
        
        // 判斷是否有左眼位置
        if(faceFeature.hasLeftEyePosition){
            NSLog(@"檢測到左眼");
        }
        // 判斷是否有右眼位置
        if(faceFeature.hasRightEyePosition){
            NSLog(@"檢測到右眼");
        }
        // 判斷是否有嘴位置
        if(faceFeature.hasMouthPosition){
            NSLog(@"檢測到嘴部");
        }
    }
}

#pragma mark -  AVCaptureVideoDataOutputSampleBufferDelegate

- (void)captureOutput:(AVCaptureFileOutput *)output didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection
{
    NSLog(@"----------");

    [connection setVideoOrientation:AVCaptureVideoOrientationPortrait];
    CVImageBufferRef buffer;
    buffer = CMSampleBufferGetImageBuffer(sampleBuffer);
    
    CVPixelBufferLockBaseAddress(buffer, 0);
    uint8_t *base;
    size_t width, height, bytesPerRow;
    base = (uint8_t *)CVPixelBufferGetBaseAddress(buffer);
    width = CVPixelBufferGetWidth(buffer);
    height = CVPixelBufferGetHeight(buffer);
    bytesPerRow = CVPixelBufferGetBytesPerRow(buffer);
    
    CGColorSpaceRef colorSpace;
    CGContextRef cgContext;
    colorSpace = CGColorSpaceCreateDeviceRGB();
    cgContext = CGBitmapContextCreate(base, width, height, 8, bytesPerRow, colorSpace, kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst);
    CGColorSpaceRelease(colorSpace);
    
    CGImageRef cgImage;
    UIImage *image;
    cgImage = CGBitmapContextCreateImage(cgContext);
    image = [UIImage imageWithCGImage:cgImage];
    [self detectFaceWithImage:image];
    CGImageRelease(cgImage);
    CGContextRelease(cgContext);
    
    CVPixelBufferUnlockBaseAddress(buffer, 0);
}

@end

下面看一下部分輸出

2018-01-31 14:18:07.001789+0800 JJFaceDetector_demo2[4700:1444754] ----------
2018-01-31 14:18:07.168074+0800 JJFaceDetector_demo2[4700:1444754] 檢測到左眼
2018-01-31 14:18:07.168400+0800 JJFaceDetector_demo2[4700:1444754] 檢測到右眼
2018-01-31 14:18:07.168557+0800 JJFaceDetector_demo2[4700:1444754] 檢測到嘴部

2018-01-31 14:18:07.174485+0800 JJFaceDetector_demo2[4700:1444754] ----------
2018-01-31 14:18:07.388472+0800 JJFaceDetector_demo2[4700:1444754] 檢測到左眼
2018-01-31 14:18:07.389386+0800 JJFaceDetector_demo2[4700:1444754] 檢測到右眼
2018-01-31 14:18:07.389440+0800 JJFaceDetector_demo2[4700:1444754] 檢測到嘴部

2018-01-31 14:18:07.398383+0800 JJFaceDetector_demo2[4700:1444754] ----------
2018-01-31 14:18:07.587945+0800 JJFaceDetector_demo2[4700:1444754] 檢測到左眼
2018-01-31 14:18:07.588429+0800 JJFaceDetector_demo2[4700:1444754] 檢測到右眼
2018-01-31 14:18:07.588796+0800 JJFaceDetector_demo2[4700:1444754] 檢測到嘴部

... ... 

下面看一下識別的效果

這個是我自己,不露臉了怕嚇到諸位,不過還是可以識別的
另外一個手機中的范爺圖片
移動另外一個手機中的范爺圖片

幾個需要說明的問題

1. info.plist文件添加key

這個簡單的說一下就可以了,iOS 10以后,相機權限需要增加key了。

2. 性能問題

移動的時候如果移動過快會有檢測不準確的現(xiàn)象,這個是由于,識別和計算臉部位置并進行標記,但是計算好如果正好進行了移動,那么標記的可能還是上一幀的位置,所有有時候標記不那么準確。

3. 部分代碼說明

先說一下這一句代碼,假如不添加下面這句代碼

self.captureMovieFileOutput.videoSettings = [NSDictionary dictionaryWithObjectsAndKeys:[NSNumber numberWithUnsignedInt:kCVPixelFormatType_32BGRA], (id)kCVPixelBufferPixelFormatTypeKey, nil];

我們運行下,看輸出

2018-01-31 14:32:57.312082+0800 JJFaceDetector_demo2[4706:1448810] ----------
2018-01-31 14:32:57.312320+0800 JJFaceDetector_demo2[4706:1448810] [Unknown process name] CGBitmapContextCreate: invalid data bytes/row: should be at least 2880 for 8 integer bits/component, 3 components, kCGImageAlphaPremultipliedFirst.
2018-01-31 14:32:57.312431+0800 JJFaceDetector_demo2[4706:1448810] [Unknown process name] CGBitmapContextCreateImage: invalid context 0x0. If you want to see the backtrace, please set CG_CONTEXT_SHOW_BACKTRACE environmental variable.
2018-01-31 14:32:57.312468+0800 JJFaceDetector_demo2[4706:1448810] [api] -[CIImage initWithCGImage:options:] failed because the CGImage is nil.

這里提示的意思是CGBitmapContextCreate創(chuàng)建上下文和圖像失敗了,是一個無效的數(shù)據(jù)位,我在stackOverFlow中找到了答案,有人和我碰到了一樣的問題。

看一下別人的Answers

Your best bet will be to set the capture video data output's videoSettings to a dictionary that specifies the pixel format you want, which you'll need to set to some variation on RGB that CGBitmapContext can handle.
The documentation has a list of all of the pixel formats that Core Video can process. Only a tiny subset of those are supported by CGBitmapContext. The format that the code you found on the internet is expecting is kCVPixelFormatType_32BGRA, but that might have been written for Macs—on iOS devices, kCVPixelFormatType_32ARGB (big-endian) might be faster. Try them both, on the device, and compare frame rates.

下面我給大家翻譯下

您最好的選擇是將捕獲視頻數(shù)據(jù)輸出的videoSettings設置為一個字典,該字典指定了您想要的像素格式,您需要在CGBitmapContext可以處理的RGB上設置一些變量。
文檔中列出了a list of all of the pixel formats that Core Video can processCGBitmapContext僅支持其中的一小部分。 您在互聯(lián)網(wǎng)上找到的代碼的格式是kCVPixelFormatType_32BGRA,但可能已經(jīng)為iOS設備上的Mac編寫,kCVPixelFormatType_32ARGB(big-endian)可能會更快。 在設備上試用它們,并比較幀速率。

所以加上上面那個setting字典就解決了問題。

后記

本篇已結束,后面更精彩~~~

?著作權歸作者所有,轉載或內(nèi)容合作請聯(lián)系作者
平臺聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點,簡書系信息發(fā)布平臺,僅提供信息存儲服務。

推薦閱讀更多精彩內(nèi)容