建筑学网站推荐,哪些网站有设计缺点,做自适应网站对设计稿的要求,大连建设工程集团有限公司WebRTC音视频通话-实现GPUImage视频美颜滤镜效果
在WebRTC音视频通话的GPUImage美颜效果图如下 可以看下 之前搭建ossrs服务#xff0c;可以查看#xff1a;https://blog.csdn.net/gloryFlow/article/details/132257196 之前实现iOS端调用ossrs音视频通话#xff0c;可以查…WebRTC音视频通话-实现GPUImage视频美颜滤镜效果
在WebRTC音视频通话的GPUImage美颜效果图如下 可以看下 之前搭建ossrs服务可以查看https://blog.csdn.net/gloryFlow/article/details/132257196 之前实现iOS端调用ossrs音视频通话可以查看https://blog.csdn.net/gloryFlow/article/details/132262724 之前WebRTC音视频通话高分辨率不显示画面问题可以查看https://blog.csdn.net/gloryFlow/article/details/132262724 修改SDP中的码率Bitrate可以查看https://blog.csdn.net/gloryFlow/article/details/132263021
一、GPUImage是什么
GPUImage是iOS上一个基于OpenGL进行图像处理的开源框架,内置大量滤镜,架构灵活,可以在其基础上很轻松地实现各种图像处理功能。
GPUImage中包含各种滤镜这里我不会使用那么多使用的是GPUImageLookupFilter及GPUImagePicture
GPUImage中有一个专门针对lookup table进行处理的滤镜函数GPUImageLookupFilter使用这个函数就可以直接对图片进行滤镜添加操作了。代码如下
/**GPUImage中有一个专门针对lookup table进行处理的滤镜函数GPUImageLookupFilter使用这个函数就可以直接对图片进行滤镜添加操作了。originalImg是你希望添加滤镜的原始图片param image 原图return 处理后的图片*/(UIImage *)applyLookupFilter:(UIImage *)image lookUpImage:(UIImage *)lookUpImage {if (lookUpImage nil) {return image;}UIImage *inputImage image;UIImage *outputImage nil;GPUImagePicture *stillImageSource [[GPUImagePicture alloc] initWithImage:inputImage];//添加滤镜GPUImageLookupFilter *lookUpFilter [[GPUImageLookupFilter alloc] init];//导入之前保存的NewLookupTable.png文件GPUImagePicture *lookupImg [[GPUImagePicture alloc] initWithImage:lookUpImage];[lookupImg addTarget:lookUpFilter atTextureLocation:1];[stillImageSource addTarget:lookUpFilter atTextureLocation:0];[lookUpFilter useNextFrameForImageCapture];if([lookupImg processImageWithCompletionHandler:nil] [stillImageSource processImageWithCompletionHandler:nil]) {outputImage [lookUpFilter imageFromCurrentFramebuffer];}return outputImage;
}这个需要lookUpImage图列表如下 由于暂时没有整理demo的git
这里在使用applyLomofiFilter再试下效果
SDApplyFilter.m中的几个方法 (UIImage *)applyBeautyFilter:(UIImage *)image {GPUImageBeautifyFilter *filter [[GPUImageBeautifyFilter alloc] init];[filter forceProcessingAtSize:image.size];GPUImagePicture *pic [[GPUImagePicture alloc] initWithImage:image];[pic addTarget:filter];[pic processImage];[filter useNextFrameForImageCapture];return [filter imageFromCurrentFramebuffer];
}/**Amatorka滤镜 Rise滤镜,可以使人像皮肤得到很好的调整param image imagereturn 处理后的图片*/(UIImage *)applyAmatorkaFilter:(UIImage *)image
{GPUImageAmatorkaFilter *filter [[GPUImageAmatorkaFilter alloc] init];[filter forceProcessingAtSize:image.size];GPUImagePicture *pic [[GPUImagePicture alloc] initWithImage:image];[pic addTarget:filter];[pic processImage];[filter useNextFrameForImageCapture];return [filter imageFromCurrentFramebuffer];
}/**复古型滤镜,感觉像旧上海滩param image imagereturn 处理后的图片*/(UIImage *)applySoftEleganceFilter:(UIImage *)image
{GPUImageSoftEleganceFilter *filter [[GPUImageSoftEleganceFilter alloc] init];[filter forceProcessingAtSize:image.size];GPUImagePicture *pic [[GPUImagePicture alloc] initWithImage:image];[pic addTarget:filter];[pic processImage];[filter useNextFrameForImageCapture];return [filter imageFromCurrentFramebuffer];
}/**图像黑白化并有大量噪点param image 原图return 处理后的图片*/(UIImage *)applyLocalBinaryPatternFilter:(UIImage *)image
{GPUImageLocalBinaryPatternFilter *filter [[GPUImageLocalBinaryPatternFilter alloc] init];[filter forceProcessingAtSize:image.size];GPUImagePicture *pic [[GPUImagePicture alloc] initWithImage:image];[pic addTarget:filter];[pic processImage];[filter useNextFrameForImageCapture];return [filter imageFromCurrentFramebuffer];
}/**单色滤镜param image 原图return 处理后的图片*/(UIImage *)applyMonochromeFilter:(UIImage *)image
{GPUImageMonochromeFilter *filter [[GPUImageMonochromeFilter alloc] init];[filter forceProcessingAtSize:image.size];GPUImagePicture *pic [[GPUImagePicture alloc] initWithImage:image];[pic addTarget:filter];[pic processImage];[filter useNextFrameForImageCapture];return [filter imageFromCurrentFramebuffer];
}使用GPUImageSoftEleganceFilter复古型滤镜,感觉像旧上海滩效果图如下
使用GPUImageLocalBinaryPatternFilter图像黑白化效果图如下 使用GPUImageMonochromeFilter 效果图如下 二、WebRTC实现音视频通话中视频滤镜处理
之前实现iOS端调用ossrs音视频通话可以查看https://blog.csdn.net/gloryFlow/article/details/132262724 这个已经有完整的代码了这里暂时做一下调整。
为RTCCameraVideoCapturer的delegate指向代理
- (RTCVideoTrack *)createVideoTrack {RTCVideoSource *videoSource [self.factory videoSource];self.localVideoSource videoSource;// 如果是模拟器if (TARGET_IPHONE_SIMULATOR) {if (available(iOS 10, *)) {self.videoCapturer [[RTCFileVideoCapturer alloc] initWithDelegate:self];} else {// Fallback on earlier versions}} else{self.videoCapturer [[RTCCameraVideoCapturer alloc] initWithDelegate:self];}RTCVideoTrack *videoTrack [self.factory videoTrackWithSource:videoSource trackId:video0];return videoTrack;
}实现RTCVideoCapturerDelegate的方法didCaptureVideoFrame
#pragma mark - RTCVideoCapturerDelegate处理代理
- (void)capturer:(RTCVideoCapturer *)capturer didCaptureVideoFrame:(RTCVideoFrame *)frame {
// DebugLog(capturer:% didCaptureVideoFrame:%, capturer, frame);// 调用SDWebRTCBufferFliter的滤镜处理RTCVideoFrame *aFilterVideoFrame;if (self.delegate [self.delegate respondsToSelector:selector(webRTCClient:didCaptureVideoFrame:)]) {aFilterVideoFrame [self.delegate webRTCClient:self didCaptureVideoFrame:frame];}// 操作C 需要手动释放 否则内存暴涨
// CVPixelBufferRelease(_buffer)// 拿到pixelBuffer
// ((RTCCVPixelBuffer*)frame.buffer).pixelBufferif (!aFilterVideoFrame) {aFilterVideoFrame frame;}[self.localVideoSource capturer:capturer didCaptureVideoFrame:frame];
}之后调用SDWebRTCBufferFliter实现滤镜效果。 实现将((RTCCVPixelBuffer *)frame.buffer).pixelBuffer进行渲染这里用到了EAGLContext、CIContext
EAGLContext是OpenGL绘制句柄或者上下文在绘制试图之前需要指定使用创建的上下文绘制。 CIContext是用来渲染CIImage,将作用在CIImage上的滤镜链应用到原始的图片数据中。我这里需要将UIImage转换成CIImage。
具体代码实现如下
SDWebRTCBufferFliter.h
#import Foundation/Foundation.h
#import WebRTCClient.hinterface SDWebRTCBufferFliter : NSObject- (RTCVideoFrame *)webRTCClient:(WebRTCClient *)client didCaptureVideoFrame:(RTCVideoFrame *)frame;endSDWebRTCBufferFliter.m
#import SDWebRTCBufferFliter.h
#import VideoToolbox/VideoToolbox.h
#import SDApplyFilter.hinterface SDWebRTCBufferFliter ()
// 滤镜
property (nonatomic, strong) EAGLContext *eaglContext;property (nonatomic, strong) CIContext *coreImageContext;property (nonatomic, strong) UIImage *lookUpImage;endimplementation SDWebRTCBufferFliter- (instancetype)init
{self [super init];if (self) {self.eaglContext [[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES3];self.coreImageContext [CIContext contextWithEAGLContext:self.eaglContext options:nil];self.lookUpImage [UIImage imageNamed:lookup_jiari];}return self;
}- (RTCVideoFrame *)webRTCClient:(WebRTCClient *)client didCaptureVideoFrame:(RTCVideoFrame *)frame {CVPixelBufferRef pixelBufferRef ((RTCCVPixelBuffer *)frame.buffer).pixelBuffer;// CFRetain(pixelBufferRef);if (pixelBufferRef) {CIImage *inputImage [CIImage imageWithCVPixelBuffer:pixelBufferRef];CGImageRef imgRef [_coreImageContext createCGImage:inputImage fromRect:[inputImage extent]];UIImage *fromImage nil;if (!fromImage) {fromImage [UIImage imageWithCGImage:imgRef];}UIImage *toImage;toImage [SDApplyFilter applyMonochromeFilter:fromImage];
//
// if (toImage nil) {
// toImage [SDApplyFilter applyLookupFilter:fromImage lookUpImage:self.lookUpImage];
// } else {
// toImage [SDApplyFilter applyLookupFilter:fromImage lookUpImage:self.lookUpImage];
// }if (toImage nil) {toImage fromImage;}CGImageRef toImgRef toImage.CGImage;CIImage *ciimage [CIImage imageWithCGImage:toImgRef];[_coreImageContext render:ciimage toCVPixelBuffer:pixelBufferRef];CGImageRelease(imgRef);//必须释放fromImage nil;toImage nil;ciimage nil;inputImage nil;}RTCCVPixelBuffer *rtcPixelBuffer [[RTCCVPixelBuffer alloc] initWithPixelBuffer:pixelBufferRef];RTCVideoFrame *filteredFrame [[RTCVideoFrame alloc] initWithBuffer:rtcPixelBufferrotation:frame.rotationtimeStampNs:frame.timeStampNs];return filteredFrame;
}end至此可以看到在WebRTC音视频通话中GPUImage视频美颜滤镜的具体效果了。
三、小结
WebRTC音视频通话-实现GPUImage视频美颜滤镜效果。主要用到GPUImage处理视频画面CVPixelBufferRef将处理后的CVPixelBufferRef生成RTCVideoFrame通过调用localVideoSource中实现的didCaptureVideoFrame方法。内容较多描述可能不准确请见谅。
本文地址https://blog.csdn.net/gloryFlow/article/details/132265842
学习记录每天不停进步。
