一、人脸图像特征提取的方法

1.HOG 特征

方向梯度直方图（Histogram of Oriented Gradient, HOG）特征是一种在计算机视觉和图 像处理中用来进行物体检测的特征描述子。它通过计算和统计图像局部区域的梯度方向 直方图来构成特征。Hog 特征结合 SVM 分类器已经被广泛应用于图像识别中，尤其在 行人检测中获得了极大的成功。需要提醒的是，HOG+SVM 进行行人检测的方法是法国 研究人员 Dalal 在 2005 的 CVPR 上提出的，而如今虽然有很多行人检测算法不断提出， 但基本都是以 HOG+SVM 的思路为主。
（1）主要思想： 在一副图像中，局部目标的表象和形状（appearance and shape）能够被梯度或边缘的 方向密度分布很好地描述。（本质：梯度的统计信息，而梯度主要存在于边缘的地方）。
（2）具体的实现方法是： 首先将图像分成小的连通区域，我们把它叫细胞单元。然后采集细胞单元中各像素点的 梯度的或边缘的方向直方图。最后把这些直方图组合起来就可以构成特征描述器。
（3）提高性能： 把这些局部直方图在图像的更大的范围内（我们把它叫区间或 block）进行对比度归一 化（contrast-normalized），所采用的方法是：先计算各直方图在这个区间（block）中 的密度，然后根据这个密度对区间中的各个细胞单元做归一化。通过这个归一化后，能 对光照变化和阴影获得更好的效果。
（4）优点： 与其他的特征描述方法相比，HOG 有很多优点。首先，由于 HOG 是在图像的局部方格 单元上操作，所以它对图像几何的和光学的形变都能保持很好的不变性，这两种形变只 会出现在更大的空间领域上。其次，在粗的空域抽样、精细的方向抽样以及较强的局部 光学归一化等条件下，只要行人大体上能够保持直立的姿势，可以容许行人有一些细微 的肢体动作，这些细微的动作可以被忽略而不影响检测效果。因此 HOG 特征是特别适 合于做图像中的人体检测的。
HOG 特征提取方法就是将一个 image（你要检测的目标或者扫描窗口）：
1）灰度化（将图像看做一个 x,y,z（灰度）的三维图像）；
2）采用 Gamma 校正法对输入图像进行颜色空间的标准化（归一化）；目的是调节图 像的对比度，降低图像局部的阴影和光照变化所造成的影响，同时可以抑制噪音的干扰；
3）计算图像每个像素的梯度（包括大小和方向）；主要是为了捕获轮廓信息，同时进 一步弱化光照的干扰。
4）将图像划分成小 cells（例如 66 像素/cell）；
5）统计每个 cell 的梯度直方图（不同梯度的个数），即可形成每个 cell 的 descriptor；
6）将每几个 cell 组成一个 block（例如 33 个 cell/block），一个 block 内所有 cell 的 特征 descriptor 串联起来便得到该 block 的 HOG 特征 descriptor。 7）将图像 image 内的所有 block 的 HOG 特征 descriptor 串联起来就可以得到该 image （你要检测的目标）的 HOG 特征 descriptor 了。这个就是最终的可供分类使用的特征 向量了。

2.Dlib

Dlib 是一个包含机器学习算法的 C++开源工具包。Dlib 可以帮助您创建很多复杂的机器 学习方面的软件来帮助解决实际问题。目前 Dlib 已经被广泛的用在行业和学术领域，包 括机器人，嵌入式设备，移动电话和大型高性能计算环境。 Dlib 的主要特点：

1. 文档齐全 不像很多其他的开源库一样，Dlib 为每一个类和函数提供了完整的文档说明。同时，还提供了 debug 模式；打开 debug 模式后，用户可以调试代码，查看变量和对象的值，快速定位错误点。 另外，Dlib 还提供了大量的实例。
2. 高质量的可移植代码 Dlib 不依赖第三方库，无须安装和配置，这部分可参照（官网左侧树形目录的 how to compile 的介绍）。Dlib 可用在 window、Mac OS、Linux 系统上。
3. 提供大量的机器学习 / 图像处理算法
4. 深度学习
   基于 SVM 的分类和递归算法 针对大规模分类和递归的降维方法 相关向量机(relevance vector machine)；是与支持向量机相同的函数形式稀疏概率模型，对未知函数进行 预测或分类。其训练是在贝叶斯框架下进行的，与 SVM 相比，不需要估计正则化参数，其核函数也不需 要满足 Mercer 条件，需要更少的相关向量，训练时间长，测试时间短。
   聚类： linear or kernel k-means, Chinese Whispers, and Newman clustering. Radial Basis Function Networks 多层感知机

3.卷积神经网络

卷积神经网络（Convolutional Neural Networks）是一种深度学习模型或类似于人工神 经网络的多层感知器，常用来分析视觉图像。卷积神经网络的创始人是着名的计算机科 学家 Yann LeCun，目前在 Facebook 工作，他是第一个通过卷积神经网络在 MNIST 数 据集上解决手写数字问题的人。 卷积神经网络 CNN 主要用来识别位移、缩放及其他形式扭曲不变性的二维图形。由于 CNN 的特征检测层通过训练数据进行学习，所以在使用 CNN 时，避免了显式的特征抽 取，而隐式地从训练数据中进行学习；再者由于同一特征映射面上的神经元权值相同， 所以网络可以并行学习，这也是卷积网络相对于神经元彼此相连网络的一大优势。卷积 神经网络以其局部权值共享的特殊结构在语音识别和图像处理方面有着独特的优越性， 其布局更接近于实际的生物神经网络，权值共享降低了网络的复杂性，特别是多维输入 向量的图像可以直接输入网络这一特点避免了特征提取和分类过程中数据重建的复杂 度。卷积神经网络中，第一步一般用卷积核去提取特征，这些初始化的卷积核会在反向传播 的过程中，在迭代中被一次又一次的更新，无限地逼近我们的真实解。其实本质没有对 图像矩阵求解，而是初始化了一个符合某种分布的特征向量集，然后在反向传播中无限 更新这个特征集，让它能无限逼近数学中的那个概念上的特征向量，以致于我们能用特 征向量的数学方法对矩阵进行特征提取。

Dlib笑脸检测

利用dilb提取嘴部特征并写入data.csv

筛选数据

# 筛选数据将无法提取特征点的图片提取出来
def select():for i in imgs_smiles:img_rd = path_images_with_smiles + i# read img fileimg = cv2.imread(img_rd)# 取灰度img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)faces = detector(img_gray, 0)try:np.matrix([[p.x, p.y] for p in predictor(img, faces[0]).parts()])except:shutil.move(img_rd,select_smiles)for i in imgs_no_smiles:img_rd = path_images_no_smiles + i# read img fileimg = cv2.imread(img_rd)# 取灰度img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)faces = detector(img_gray, 0)try:np.matrix([[p.x, p.y] for p in predictor(img, faces[0]).parts()])except:shutil.move(img_rd,select_no_smiles)

提取特征点

# 输入图像文件所在路径，返回一个41维数组（包含提取到的40维特征和1维输出标记）
def get_features(img_rd):# 输入:  img_rd:      图像文件# 输出:  positions_lip_arr:  feature point 49 to feature point 68, 20 feature points / 40D in all# read img fileimg = cv2.imread(img_rd)# 取灰度img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)# 计算68点坐标positions_68_arr = []faces = detector(img_gray, 0)landmarks = np.matrix([[p.x, p.y] for p in predictor(img, faces[0]).parts()])# except:for point in enumerate(landmarks):# 68点的坐标pos = (point[0, 0], point[0, 1])positions_68_arr.append(pos)positions_lip_arr = []# 将点 49-68 写入 CSV# 即 positions_68_arr[48]-positions_68_arr[67]for i in range(48, 68):positions_lip_arr.append(positions_68_arr[i][0])positions_lip_arr.append(positions_68_arr[i][1])return positions_lip_arr

完整代码

import dlib         # 人脸处理的库 Dlib
import numpy as np  # 数据处理的库 numpy
from cv2 import cv2          # 图像处理的库 OpenCv
import os           # 读取文件
import csv          # CSV 操作
import shutildetector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor('D:\\Desktop\\Face_recognition\\shape_predictor_68_face_landmarks.dat')# 输入图像文件所在路径，返回一个41维数组（包含提取到的40维特征和1维输出标记）
def get_features(img_rd):# 输入:  img_rd:      图像文件# 输出:  positions_lip_arr:  feature point 49 to feature point 68, 20 feature points / 40D in all# read img fileimg = cv2.imread(img_rd)# 取灰度img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)# 计算68点坐标positions_68_arr = []faces = detector(img_gray, 0)landmarks = np.matrix([[p.x, p.y] for p in predictor(img, faces[0]).parts()])# except:for idx, point in enumerate(landmarks):# 68点的坐标pos = (point[0, 0], point[0, 1])positions_68_arr.append(pos)positions_lip_arr = []# 将点 49-68 写入 CSV# 即 positions_68_arr[48]-positions_68_arr[67]for i in range(48, 68):positions_lip_arr.append(positions_68_arr[i][0])positions_lip_arr.append(positions_68_arr[i][1])return positions_lip_arr# 读取图像所在的路径
path_images_with_smiles = "D:\\Desktop\\Face_recognition\\data\\data_imgs\\database\\smiles\\"
path_images_no_smiles = "D:\\Desktop\\Face_recognition\\data\\data_imgs\\database\\no_smiles\\"# 获取路径下的图像文件
imgs_smiles = os.listdir(path_images_with_smiles)
imgs_no_smiles = os.listdir(path_images_no_smiles)
select_smiles = "D:\\Desktop\\Face_recognition\\data\\data_imgs\\database\\select\\1\\"
select_no_smiles = "D:\\Desktop\\Face_recognition\\data\\data_imgs\\database\\select\\2\\"
# 存储提取特征数据的 CSV 的路径
path_csv = "data\\data_csvs\\"# 筛选数据将无法提取特征点的图片提取出来
def select():for i in imgs_smiles:img_rd = path_images_with_smiles + i# read img fileimg = cv2.imread(img_rd)# 取灰度img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)faces = detector(img_gray, 0)try:np.matrix([[p.x, p.y] for p in predictor(img, faces[0]).parts()])except:shutil.move(img_rd,select_smiles)for i in imgs_no_smiles:img_rd = path_images_no_smiles + i# read img fileimg = cv2.imread(img_rd)# 取灰度img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)faces = detector(img_gray, 0)try:np.matrix([[p.x, p.y] for p in predictor(img, faces[0]).parts()])except:shutil.move(img_rd,select_no_smiles)# write the features into CSV
def write_into_CSV():with open(path_csv+"data.csv", "w", newline="") as csvfile:writer = csv.writer(csvfile)# 处理带笑脸的图像print("######## with smiles #########")for i in imgs_smiles:print(path_images_with_smiles + i)features_csv_smiles = get_features(path_images_with_smiles+ i)features_csv_smiles.append(1)print("positions of lips:", features_csv_smiles, "\n")# 写入CSVwriter.writerow(features_csv_smiles)# 处理不带笑脸的图像print("######## no smiles #########")for i in imgs_no_smiles:print(path_images_no_smiles + i)features_csv_no_smiles = get_features(path_images_no_smiles + i)features_csv_no_smiles.append(0)print("positions of lips:", features_csv_no_smiles, "\n")# 写入CSVwriter.writerow(features_csv_no_smiles)# 数据筛选
# select()# 写入CSV
# write_into_CSV()

利用sklearn提取训练集X_train和测试集X_test

完整代码

# pandas 读取 CSV
import pandas as pd# 分割数据
from sklearn.model_selection import train_test_split# 用于数据预加工标准化
from sklearn.preprocessing import StandardScalerfrom sklearn.linear_model import LogisticRegression     # 线性模型中的 Logistic 回归模型
from sklearn.neural_network import MLPClassifier        # 神经网络模型中的多层网络模型
from sklearn.svm import LinearSVC                       # SVM 模型中的线性 SVC 模型
from sklearn.linear_model import SGDClassifier          # 线性模型中的随机梯度下降模型import joblib# 从 csv 读取数据
def pre_data():# 41 维表头column_names = []for i in range(0, 40):column_names.append("feature_" + str(i + 1))column_names.append("output")# read csvrd_csv = pd.read_csv("data/data_csvs/data.csv", names=column_names)# 输出 csv 文件的维度# print("shape:", rd_csv.shape)X_train, X_test, y_train, y_test = train_test_split(# input 0-40# output 41rd_csv[column_names[0:40]],rd_csv[column_names[40]],# 25% for testing, 75% for trainingtest_size=0.25,random_state=33)return X_train, X_test, y_train, y_testpath_models = "data/data_models/"# LR, logistic regression, 逻辑斯特回归分类（线性模型）
def model_LR():# get dataX_train_LR, X_test_LR, y_train_LR, y_test_LR = pre_data()# 数据预加工# 标准化数据，保证每个维度的特征数据方差为1，均值为0。使得预测结果不会被某些维度过大的特征值而主导ss_LR = StandardScaler()X_train_LR = ss_LR.fit_transform(X_train_LR)X_test_LR = ss_LR.transform(X_test_LR)# 初始化 LogisticRegressionLR = LogisticRegression()# 调用 LogisticRegression 中的 fit() 来训练模型参数LR.fit(X_train_LR, y_train_LR)# save LR modeljoblib.dump(LR, path_models + "model_LR.m")# 评分函数score_LR = LR.score(X_test_LR, y_test_LR)print("The accurary of LR:", score_LR)# print(type(ss_LR))return (ss_LR)# model_LR()# MLPC, Multi-layer Perceptron Classifier, 多层感知机分类（神经网络）
def model_MLPC():# get dataX_train_MLPC, X_test_MLPC, y_train_MLPC, y_test_MLPC = pre_data()# 数据预加工ss_MLPC = StandardScaler()X_train_MLPC = ss_MLPC.fit_transform(X_train_MLPC)X_test_MLPC = ss_MLPC.transform(X_test_MLPC)# 初始化 MLPCMLPC = MLPClassifier(hidden_layer_sizes=(13, 13, 13), max_iter=500)# 调用 MLPC 中的 fit() 来训练模型参数MLPC.fit(X_train_MLPC, y_train_MLPC)# save MLPC modeljoblib.dump(MLPC, path_models + "model_MLPC.m")# 评分函数score_MLPC = MLPC.score(X_test_MLPC, y_test_MLPC)print("The accurary of MLPC:", score_MLPC)return (ss_MLPC)# model_MLPC()# Linear SVC， Linear Supported Vector Classifier, 线性支持向量分类(SVM支持向量机)
def model_LSVC():# get dataX_train_LSVC, X_test_LSVC, y_train_LSVC, y_test_LSVC = pre_data()# 数据预加工ss_LSVC = StandardScaler()X_train_LSVC = ss_LSVC.fit_transform(X_train_LSVC)X_test_LSVC = ss_LSVC.transform(X_test_LSVC)# 初始化 LSVCLSVC = LinearSVC()# 调用 LSVC 中的 fit() 来训练模型参数LSVC.fit(X_train_LSVC, y_train_LSVC)# save LSVC modeljoblib.dump(LSVC, path_models + "model_LSVC.m")# 评分函数score_LSVC = LSVC.score(X_test_LSVC, y_test_LSVC)print("The accurary of LSVC:", score_LSVC)return ss_LSVC# model_LSVC()# SGDC, Stochastic Gradient Decent Classifier, 随机梯度下降法求解(线性模型)
def model_SGDC():# get dataX_train_SGDC, X_test_SGDC, y_train_SGDC, y_test_SGDC = pre_data()# 数据预加工ss_SGDC = StandardScaler()X_train_SGDC = ss_SGDC.fit_transform(X_train_SGDC)X_test_SGDC = ss_SGDC.transform(X_test_SGDC)# 初始化 SGDCSGDC = SGDClassifier(max_iter=5)# 调用 SGDC 中的 fit() 来训练模型参数SGDC.fit(X_train_SGDC, y_train_SGDC)# save SGDC modeljoblib.dump(SGDC, path_models + "model_SGDC.m")# 评分函数score_SGDC = SGDC.score(X_test_SGDC, y_test_SGDC)print("The accurary of SGDC:", score_SGDC)return ss_SGDC# model_SGDC()

模型测试

完整代码

# use the saved model
from sklearn.externals import joblibfrom get_features import get_features
import ML_ways_sklearnfrom cv2 import cv2# path of test img
path_test_img = "data/data_imgs/test_imgs/sunxiaoc.jpg"# 提取单张40维度特征
positions_lip_test = get_features(path_test_img)# path of models
path_models = "data/data_models/"print("The result of"+path_test_img+":")
print('\n')# #########  LR  ###########
LR = joblib.load(path_models+"model_LR.m")
ss_LR = ML_ways_sklearn.model_LR()
X_test_LR = ss_LR.transform([positions_lip_test])
y_predict_LR = str(LR.predict(X_test_LR)[0]).replace('0', "no smile").replace('1', "with smile")
print("LR:", y_predict_LR)# #########  LSVC  ###########
LSVC = joblib.load(path_models+"model_LSVC.m")
ss_LSVC = ML_ways_sklearn.model_LSVC()
X_test_LSVC = ss_LSVC.transform([positions_lip_test])
y_predict_LSVC = str(LSVC.predict(X_test_LSVC)[0]).replace('0', "no smile").replace('1', "with smile")
print("LSVC:", y_predict_LSVC)# #########  MLPC  ###########
MLPC = joblib.load(path_models+"model_MLPC.m")
ss_MLPC = ML_ways_sklearn.model_MLPC()
X_test_MLPC = ss_MLPC.transform([positions_lip_test])
y_predict_MLPC = str(MLPC.predict(X_test_MLPC)[0]).replace('0', "no smile").replace('1', "with smile")
print("MLPC:", y_predict_MLPC)# #########  SGDC  ###########
SGDC = joblib.load(path_models+"model_SGDC.m")
ss_SGDC = ML_ways_sklearn.model_SGDC()
X_test_SGDC = ss_SGDC.transform([positions_lip_test])
y_predict_SGDC = str(SGDC.predict(X_test_SGDC)[0]).replace('0', "no smile").replace('1', "with smile")
print("SGDC:", y_predict_SGDC)img_test = cv2.imread(path_test_img)img_height = int(img_test.shape[0])
img_width = int(img_test.shape[1])# show the results on the image
font = cv2.FONT_HERSHEY_SIMPLEX
cv2.putText(img_test, "LR:    "+y_predict_LR,   (int(img_height/10), int(img_width/10)), font, 0.5, (84, 255, 159), 1, cv2.LINE_AA)
cv2.putText(img_test, "LSVC:  "+y_predict_LSVC, (int(img_height/10), int(img_width/10*2)), font, 0.5, (84, 255, 159), 1, cv2.LINE_AA)
cv2.putText(img_test, "MLPC:  "+y_predict_MLPC, (int(img_height/10), int(img_width/10)*3), font, 0.5, (84, 255, 159), 1, cv2.LINE_AA)
cv2.putText(img_test, "SGDC:  "+y_predict_SGDC, (int(img_height/10), int(img_width/10)*4), font, 0.5, (84, 255, 159), 1, cv2.LINE_AA)cv2.namedWindow("img", 2)
cv2.imshow("img", img_test)
cv2.waitKey(0)

显示嘴部特征点

完整代码

# 显示嘴部特征点
# Draw the positions of someone's lipimport dlib         # 人脸识别的库 Dlib
from cv2 import cv2          # 图像处理的库 OpenCv
from get_features import get_features   # return the positions of feature pointspath_test_img = "data/data_imgs/test_imgs/test1.jpg"detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor('shape_predictor_68_face_landmarks.dat')# Get lip's positions of features points
positions_lip = get_features(path_test_img)img_rd = cv2.imread(path_test_img)# Draw on the lip points
for i in range(0, len(positions_lip), 2):print(positions_lip[i], positions_lip[i+1])cv2.circle(img_rd, tuple([positions_lip[i], positions_lip[i+1]]), radius=1, color=(0, 255, 0))cv2.namedWindow("img_read", 2)
cv2.imshow("img_read", img_rd)
cv2.waitKey(0)

视频检测

完整代码

# use the saved model
from sklearn.externals import joblibimport ML_ways_sklearnimport dlib         # 人脸处理的库 Dlib
import numpy as np  # 数据处理的库 numpy
from cv2 import cv2          # 图像处理的库 OpenCvdetector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor('shape_predictor_68_face_landmarks.dat')# OpenCv 调用摄像头
cap = cv2.VideoCapture(0)# 设置视频参数
cap.set(3, 480)def get_features(img_rd):# 输入:  img_rd:      图像文件# 输出:  positions_lip_arr:  feature point 49 to feature point 68, 20 feature points / 40D in all# 取灰度img_gray = cv2.cvtColor(img_rd, cv2.COLOR_RGB2GRAY)# 计算68点坐标positions_68_arr = []faces = detector(img_gray, 0)landmarks = np.matrix([[p.x, p.y] for p in predictor(img_rd, faces[0]).parts()])for idx, point in enumerate(landmarks):# 68点的坐标pos = (point[0, 0], point[0, 1])positions_68_arr.append(pos)positions_lip_arr = []# 将点 49-68 写入 CSV# 即 positions_68_arr[48]-positions_68_arr[67]for i in range(48, 68):positions_lip_arr.append(positions_68_arr[i][0])positions_lip_arr.append(positions_68_arr[i][1])return positions_lip_arrwhile cap.isOpened():# 480 height * 640 widthflag, img_rd = cap.read()kk = cv2.waitKey(1)img_gray = cv2.cvtColor(img_rd, cv2.COLOR_RGB2GRAY)# 人脸数 facesfaces = detector(img_gray, 0)# 检测到人脸if len(faces) != 0:# 提取单张40维度特征positions_lip_test = get_features(img_rd)# path of modelspath_models = "data/data_models/"# #########  LR  ###########LR = joblib.load(path_models+"model_LR.m")ss_LR = ML_ways_sklearn.model_LR()X_test_LR = ss_LR.transform([positions_lip_test])y_predict_LR = str(LR.predict(X_test_LR)[0]).replace('0', "no smile").replace('1', "with smile")print("LR:", y_predict_LR)# #########  LSVC  ###########LSVC = joblib.load(path_models+"model_LSVC.m")ss_LSVC = ML_ways_sklearn.model_LSVC()X_test_LSVC = ss_LSVC.transform([positions_lip_test])y_predict_LSVC = str(LSVC.predict(X_test_LSVC)[0]).replace('0', "no smile").replace('1', "with smile")print("LSVC:", y_predict_LSVC)# #########  MLPC  ###########MLPC = joblib.load(path_models+"model_MLPC.m")ss_MLPC = ML_ways_sklearn.model_MLPC()X_test_MLPC = ss_MLPC.transform([positions_lip_test])y_predict_MLPC = str(MLPC.predict(X_test_MLPC)[0]).replace('0', "no smile").replace('1', "with smile")print("MLPC:", y_predict_MLPC)# #########  SGDC  ###########SGDC = joblib.load(path_models+"model_SGDC.m")ss_SGDC = ML_ways_sklearn.model_SGDC()X_test_SGDC = ss_SGDC.transform([positions_lip_test])y_predict_SGDC = str(SGDC.predict(X_test_SGDC)[0]).replace('0', "no smile").replace('1', "with smile")print("SGDC:", y_predict_SGDC)print('\n')# 按下 'q' 键退出if kk == ord('q'):break# 窗口显示# cv2.namedWindow("camera", 0) # 如果需要摄像头窗口大小可调cv2.imshow("camera", img_rd)# 释放摄像头
cap.release()# 删除建立的窗口
cv2.destroyAllWindows()

安装TensorFlow（python3.7）

## 创建Anaconda虚拟环境Python3.6

conda create -n `你的命名` python=3.6

进入虚拟环境

python3.7安装TensorFlow

安装Keras

pip install keras

划分数据集

mkdir.py

import tensorflow as tf
import os,shutil
# The path to the directory where the original
# dataset was uncompressed
original_dataset_dir = 'D:\\Desktop\\Face_recognition\\tensorflow\\genki4k '# The directory where we will
# store our smaller dataset
base_dir = 'D:\\Desktop\\Face_recognition\\tensorflow\\smile_data'
os.mkdir(base_dir)# Directories for our training,
# validation and test splits
train_dir = os.path.join(base_dir, 'train')
os.mkdir(train_dir)
validation_dir = os.path.join(base_dir, 'validation')
os.mkdir(validation_dir)
test_dir = os.path.join(base_dir, 'test')
os.mkdir(test_dir)# Directory with our training smile pictures
train_smile_dir = os.path.join(train_dir, 'smile')
os.mkdir(train_smile_dir)# Directory with our training unsmile pictures
train_unsmile_dir = os.path.join(train_dir, 'unsmile')
os.mkdir(train_unsmile_dir)# Directory with our validation smile pictures
validation_smile_dir = os.path.join(validation_dir, 'smile')
os.mkdir(validation_smile_dir)# Directory with our validation unsmile pictures
validation_unsmile_dir = os.path.join(validation_dir, 'unsmile')
os.mkdir(validation_unsmile_dir)# Directory with our validation smile pictures
test_smile_dir = os.path.join(test_dir, 'smile')
os.mkdir(test_smile_dir)# Directory with our validation unsmile pictures
test_unsmile_dir = os.path.join(test_dir, 'unsmile')
os.mkdir(test_unsmile_dir)

运行结果：

将笑脸和非笑脸放入相应的文件夹
查看一下图片数量：
lenpic.py

import keras
import os, shutil
train_smile_dir="D:\\Desktop\\Face_recognition\\tensorflow\\smile_data\\train\\smile\\"
train_umsmile_dir="D:\\Desktop\\Face_recognition\\tensorflow\\smile_data\\train\\unsmile\\"
test_smile_dir="D:\\Desktop\\Face_recognition\\tensorflow\\smile_data\\test\\smile\\"
test_umsmile_dir="D:\\Desktop\\Face_recognition\\tensorflow\\smile_data\\test/unsmile\\"
validation_smile_dir="D:\\Desktop\\Face_recognition\\tensorflow\\smile_data\\validation\\smile\\"
validation_unsmile_dir="D:\\Desktop\\Face_recognition\\tensorflow\\smile_data\\validation\\unsmile\\"
train_dir="D:\\Desktop\\Face_recognition\\tensorflow\\smile_data\\train\\"
test_dir="D:\\Desktop\\Face_recognition\\tensorflow\\smile_data\\test\\"
validation_dir="D:\\Desktop\\Face_recognition\\tensorflow\\smile_data\\validation\\"
print('total training smile images:', len(os.listdir(train_smile_dir)))
print('total training unsmile images:', len(os.listdir(train_umsmile_dir)))
print('total testing smile images:', len(os.listdir(test_smile_dir)))
print('total testing unsmile images:', len(os.listdir(test_umsmile_dir)))
print('total validation smile images:', len(os.listdir(validation_smile_dir)))
print('total validation unsmile images:', len(os.listdir(validation_unsmile_dir)))

创建模型
Cmodle.py

'''
创建模型
'''
from keras import layers
from keras import models
model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu',input_shape=(150, 150, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(128, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(128, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Flatten())
model.add(layers.Dense(512, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))
print(model.summary())

对图片归一化：
normalize.py

'''
对图片进行归一化处理
'''
from keras import optimizers
from Cmodel import model
from lenpic import train_dir,validation_dir,test_dir
model.compile(loss='binary_crossentropy',optimizer=optimizers.RMSprop(lr=1e-4),metrics=['acc'])from keras.preprocessing.image import ImageDataGenerator
# All images will be rescaled by 1./255
train_datagen = ImageDataGenerator(rescale=1./255)
validation_datagen=ImageDataGenerator(rescale=1./255)
test_datagen = ImageDataGenerator(rescale=1./255)train_generator = train_datagen.flow_from_directory(# 目标文件目录train_dir,#所有图片的size必须是150x150target_size=(150, 150),batch_size=20,# Since we use binary_crossentropy loss, we need binary labelsclass_mode='binary')validation_generator = test_datagen.flow_from_directory(validation_dir,target_size=(150, 150),batch_size=20,class_mode='binary')
test_generator = test_datagen.flow_from_directory(test_dir,target_size=(150, 150),batch_size=20,class_mode='binary')
for data_batch, labels_batch in train_generator:print('data batch shape:', data_batch.shape)print('labels batch shape:', labels_batch)break
print(train_generator.class_indices)

模型训练

Tmodle.py

'''
模型训练
'''
from Cmodel import model
from normalize import train_generator,validation_generator
history = model.fit_generator(train_generator,steps_per_epoch=100,epochs=10,validation_data=validation_generator,validation_steps=50)
model.save('model.save('D:\\Desktop\\Face_recognition\\tensorflow\\smileAndUnsmile_1.h5')')'''
画出训练集与验证集的精确度与损失度的图形
'''
import matplotlib.pyplot as plt
acc = history.history['acc']
val_acc = history.history['val_acc']
loss = history.history['loss']
val_loss = history.history['val_loss']
epochs = range(len(acc))
plt.plot(epochs, acc, 'bo', label='Training acc')
plt.plot(epochs, val_acc, 'b', label='Validation acc')
plt.title('Training and validation accuracy')
plt.legend()
plt.figure()
plt.plot(epochs, loss, 'bo', label='Training loss')
plt.plot(epochs, val_loss, 'b', label='Validation loss')
plt.title('Training and validation loss')
plt.legend()
plt.show()

数据增强

endata.py

'''
数据增强
'''
from keras.preprocessing.image import ImageDataGenerator
datagen = ImageDataGenerator(rotation_range=40,width_shift_range=0.2,height_shift_range=0.2,shear_range=0.2,zoom_range=0.2,horizontal_flip=True,fill_mode='nearest')
'''
查看数据增强后的图片变化
'''
import matplotlib.pyplot as plt
from lenpic import train_smile_dir
import os
# This is module with image preprocessing utilities
from keras.preprocessing import image
fnames = [os.path.join(train_smile_dir, fname) for fname in os.listdir(train_smile_dir)]
img_path = fnames[8]
img = image.load_img(img_path, target_size=(150, 150))
x = image.img_to_array(img)
x = x.reshape((1,) + x.shape)
i = 0
for batch in datagen.flow(x, batch_size=1):plt.figure(i)imgplot = plt.imshow(image.array_to_img(batch[0]))i += 1if i % 4 == 0:break
plt.show()

创建网络

Cnet.py

'''
创建网络
'''
from keras import models
from keras import layers 
from keras import optimizers
model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu',input_shape=(150, 150, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(128, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(128, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Flatten())
model.add(layers.Dropout(0.5))
model.add(layers.Dense(512, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy',optimizer=optimizers.RMSprop(lr=1e-4),metrics=['acc'])

再次进行训练模型

T2modle.py

'''
再次训练模型
'''
#归一化处理
from keras.preprocessing.image import ImageDataGenerator
from lenpic import train_dir, validation_dir
from Cnet import model
train_datagen = ImageDataGenerator(rescale=1./255,rotation_range=40,width_shift_range=0.2,height_shift_range=0.2,shear_range=0.2,zoom_range=0.2,horizontal_flip=True,)# Note that the validation data should not be augmented!
test_datagen = ImageDataGenerator(rescale=1./255)train_generator = train_datagen.flow_from_directory(# This is the target directorytrain_dir,# All images will be resized to 150x150target_size=(150, 150),batch_size=32,# Since we use binary_crossentropy loss, we need binary labelsclass_mode='binary')validation_generator = test_datagen.flow_from_directory(validation_dir,target_size=(150, 150),batch_size=32,class_mode='binary')history = model.fit_generator(train_generator,steps_per_epoch=100,epochs=100,  # 训练次数validation_data=validation_generator,validation_steps=50)# 保存训练模型
model.save('D:\\Desktop\\Face_recognition\\tensorflow\\smileAndUnsmile_2.h5')

训练50次：

训练100次：

使用模型进行视频实时检测

'''
检测视频或者摄像头中的人脸
'''
from cv2 import cv2
from keras.preprocessing import image
from keras.models import load_model
import numpy as np
import dlib
from PIL import Image
model = load_model('D:\\Desktop\\Face_recognition\\tensorflow\\smileAndUnsmile_2.h5')
detector = dlib.get_frontal_face_detector()
video=cv2.VideoCapture(0)
font = cv2.FONT_HERSHEY_SIMPLEX
def rec(img):gray=cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)dets=detector(gray,1)if dets is not None:for face in dets:left=face.left()top=face.top()right=face.right()bottom=face.bottom()cv2.rectangle(img,(left,top),(right,bottom),(0,255,0),2)img1=cv2.resize(img[top:bottom,left:right],dsize=(150,150))img1=cv2.cvtColor(img1,cv2.COLOR_BGR2RGB)img1 = np.array(img1)/255.img_tensor = img1.reshape(-1,150,150,3)prediction =model.predict(img_tensor)    if prediction[0][0]>0.5:result='unsmile'else:result='smile'cv2.putText(img, result, (left,top), font, 2, (0, 255, 0), 2, cv2.LINE_AA)cv2.imshow('Video', img)
while video.isOpened():res, img_rd = video.read()if not res:breakrec(img_rd)if cv2.waitKey(1) & 0xFF == ord('q'):break
video.release()
cv2.destroyAllWindows()

口罩识别

划分数据集

mkdir.py

# The path to the directory where the original
# dataset was uncompressed# The directory where we will
# store our smaller dataset
base_dir = 'D:\\Desktop\\Face_recognition\\Mask\\mask_data'
os.mkdir(base_dir)# Directories for our training,
# validation and test splits
train_dir = os.path.join(base_dir, 'train')
os.mkdir(train_dir)
validation_dir = os.path.join(base_dir, 'validation')
os.mkdir(validation_dir)
test_dir = os.path.join(base_dir, 'test')
os.mkdir(test_dir)# Directory with our training mask pictures
train_mask_dir = os.path.join(train_dir, 'have_mask')
os.mkdir(train_mask_dir)# Directory with our training no_mask pictures
train_no_mask_dir = os.path.join(train_dir, 'no_mask')
os.mkdir(train_no_mask_dir)# Directory with our validation mask pictures
validation_mask_dir = os.path.join(validation_dir, 'have_mask')
os.mkdir(validation_mask_dir)# Directory with our validation no_mask pictures
validation_no_mask_dir = os.path.join(validation_dir, 'no_mask')
os.mkdir(validation_no_mask_dir)# Directory with our validation mask pictures
test_mask_dir = os.path.join(test_dir, 'have_mask')
os.mkdir(test_mask_dir)# Directory with our validation no_mask pictures
test_no_mask_dir = os.path.join(test_dir, 'no_mask')
os.mkdir(test_no_mask_dir)

添加口罩图片到相应文件夹

下载口罩数据集
下载地址
由于它是多个文件存储的，所以要将图片合并到一个文件夹
move.py

import os
outer_path = 'D:\\Desktop\\Face_recognition\\Mask\\have_mask\\RWMFD_part_1'
folderlist = os.listdir(outer_path)  
print(folderlist)        #列举文件夹
i = 0
for folder in folderlist:     inner_path = os.path.join(outer_path, folder)total_num_folder = len(folderlist)       #文件夹的总数print('total have %d folders' % (total_num_folder))    #打印文件夹的总数filelist = os.listdir(inner_path)        #列举图片for item in filelist:total_num_file = len(filelist)       #单个文件夹内图片的总数if item.endswith('.jpg'):src = os.path.join(os.path.abspath(inner_path), item)           #原图的地址dst = os.path.join(os.path.abspath(inner_path), str('D:\\Desktop\\Face_recognition\\Mask\\have_mask\\RWMFD_part_1\\MASK') + '_' + str(i) + '.jpg')        #新图的地址（这里可以把str(folder) + '_' + str(i) + '.jpg'改成你想改的名称）try:os.rename(src, dst)print('converting %s to %s ...' % (src, dst)) i += 1except:continueprint('total %d to rename & converted %d jpgs' % (total_num_file, i)) 

查看图片数量

lenpic.py

import keras
import os, shutil
train_mask_dir="D:\\Desktop\\Face_recognition\\Mask\\mask_data\\train\\have_mask\\"
train_ummask_dir="D:\\Desktop\\Face_recognition\\Mask\\mask_data\\train\\no_mask\\"
test_mask_dir="D:\\Desktop\\Face_recognition\\Mask\\mask_data\\test\\have_mask\\"
test_ummask_dir="D:\\Desktop\\Face_recognition\\Mask\\mask_data\\test\\no_mask\\"
validation_mask_dir="D:\\Desktop\\Face_recognition\\Mask\\mask_data\\validation\\have_mask\\"
validation_unmask_dir="D:\\Desktop\\Face_recognition\\Mask\\mask_data\\validation\\no_mask\\"
train_dir="D:\\Desktop\\Face_recognition\\Mask\\mask_data\\train\\"
test_dir="D:\\Desktop\\Face_recognition\\Mask\\mask_data\\test\\"
validation_dir="D:\\Desktop\\Face_recognition\\Mask\\mask_data\\validation\\"
print('total training mask images:', len(os.listdir(train_mask_dir)))
print('total training unmask images:', len(os.listdir(train_ummask_dir)))
print('total testing mask images:', len(os.listdir(test_mask_dir)))
print('total testing unmask images:', len(os.listdir(test_ummask_dir)))
print('total validation mask images:', len(os.listdir(validation_mask_dir)))
print('total validation unmask images:', len(os.listdir(validation_unmask_dir)))

数据增强

endata.py

from keras.preprocessing.image import ImageDataGenerator
datagen = ImageDataGenerator(rotation_range=40,width_shift_range=0.2,height_shift_range=0.2,shear_range=0.2,zoom_range=0.2,horizontal_flip=True,fill_mode='nearest')
'''
查看数据增强后的图片变化
'''
import matplotlib.pyplot as plt
from lenpic import train_mask_dir
import os
# This is module with image preprocessing utilities
from keras.preprocessing import image
fnames = [os.path.join(train_mask_dir, fname) for fname in os.listdir(train_mask_dir)]
img_path = fnames[8]
img = image.load_img(img_path, target_size=(150, 150))
x = image.img_to_array(img)
x = x.reshape((1,) + x.shape)
i = 0
for batch in datagen.flow(x, batch_size=1):plt.figure(i)imgplot = plt.imshow(image.array_to_img(batch[0]))i += 1if i % 4 == 0:break
plt.show()

创建网络

Cnet.py

'''
创建网络
'''
from keras import models
from keras import layers 
from keras import optimizers
model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu',input_shape=(150, 150, 3)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(128, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(128, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Flatten())
model.add(layers.Dropout(0.5))
model.add(layers.Dense(512, activation='relu'))
model.add(layers.Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy',optimizer=optimizers.RMSprop(lr=1e-4),metrics=['acc'])

模型训练 并画出训练集与验证集的精确度与损失度的图形

Tmodle.py

#归一化处理
from keras.preprocessing.image import ImageDataGenerator
from lenpic import train_dir, validation_dir
from Cnet import model
train_datagen = ImageDataGenerator(rescale=1./255,rotation_range=40,width_shift_range=0.2,height_shift_range=0.2,shear_range=0.2,zoom_range=0.2,horizontal_flip=True,)# Note that the validation data should not be augmented!
test_datagen = ImageDataGenerator(rescale=1./255)train_generator = train_datagen.flow_from_directory(# This is the target directorytrain_dir,# All images will be resized to 150x150target_size=(150, 150),batch_size=32,# Since we use binary_crossentropy loss, we need binary labelsclass_mode='binary')validation_generator = test_datagen.flow_from_directory(validation_dir,target_size=(150, 150),batch_size=32,class_mode='binary')history = model.fit_generator(train_generator,steps_per_epoch=100,epochs=100,  # 训练次数validation_data=validation_generator,validation_steps=50)# 保存训练模型
model.save('D:\\Desktop\\Face_recognition\\Mask\\maskAndUnmask_1.h5')'''
画出训练集与验证集的精确度与损失度的图形
'''
import matplotlib.pyplot as plt
acc = history.history['acc']
val_acc = history.history['val_acc']
loss = history.history['loss']
val_loss = history.history['val_loss']
epochs = range(len(acc))
plt.plot(epochs, acc, 'bo', label='Training acc')
plt.plot(epochs, val_acc, 'b', label='Validation acc')
plt.title('Training and validation accuracy')
plt.legend()
plt.figure()
plt.plot(epochs, loss, 'bo', label='Training loss')
plt.plot(epochs, val_loss, 'b', label='Validation loss')
plt.title('Training and validation loss')
plt.legend()
plt.show()

摄像头检测

camera.py

'''
检测视频或者摄像头中的人脸
'''
from cv2 import cv2from keras.preprocessing import image
from keras.models import load_model
import numpy as np
import dlib
from PIL import Image
model = load_model('D:\\Desktop\\Face_recognition\\Mask\\maskAndUnmask_1.h5')
detector = dlib.get_frontal_face_detector()
video=cv2.VideoCapture(0)
font = cv2.FONT_HERSHEY_SIMPLEX
def rec(img):gray=cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)dets=detector(gray,1)if dets is not None:for face in dets:left=face.left()top=face.top()right=face.right()bottom=face.bottom()cv2.rectangle(img,(left,top),(right,bottom),(0,255,0),2)img1=cv2.resize(img[top:bottom,left:right],dsize=(150,150))img1=cv2.cvtColor(img1,cv2.COLOR_BGR2RGB)img1 = np.array(img1)/255.img_tensor = img1.reshape(-1,150,150,3)prediction =model.predict(img_tensor)if prediction[0][0]>0.5:result='unmask'else:result='mask'cv2.putText(img, result, (left,top), font, 2, (0, 255, 0), 2, cv2.LINE_AA)cv2.imshow('mask', img)
while video.isOpened():res, img_rd = video.read()if not res:breakrec(img_rd)if cv2.waitKey(5) & 0xFF == ord('q'):break
video.release()
cv2.destroyAllWindows()

参考文献

人工智能与机器学习
Dlib模型人脸特征检测原理及demo
cungudafa的博客