视频理解论文笔记-I3D S3D R(2+1)D P3D CSN

I3D

Carreira, Joao, and Andrew Zisserman. “Quo vadis, action recognition? a new model and the kinetics dataset.” proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017.

S3D

Xie, Saining, et al. “Rethinking spatiotemporal feature learning: Speed-accuracy trade-offs in video classification.” Proceedings of the European Conference on Computer Vision (ECCV). 2018.

R(2+1)D

Tran, Du, et al. “A closer look at spatiotemporal convolutions for action recognition.” Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 2018.

P3D

Qiu, Zhaofan, Ting Yao, and Tao Mei. “Learning spatio-temporal representation with pseudo-3d residual networks.” proceedings of the IEEE International Conference on Computer Vision. 2017.

CSN

Tran, Du, et al. “Video classification with channel-separated convolutional networks.” Proceedings of the IEEE International Conference on Computer Vision. 2019.

都是基于3D卷积分解的文章

I3D 是 Zisserman

R(2+1)D是C3D作者 Tran Du，IDT作者hengwang ，养乐村在FAIR的作品

P3D

I3D Contributions:

1. Kinects 400 数据集，每类400+ Youtube视频
2. Inflating 2D ConvNets into 3D
3. Bootstrapping 3D filters from 2D Filters
4. Inception 结构将333卷积分解为111再333

S3D Contributions:

1. kt x k x k 卷积分解为 1 x k x k + kt x 1 x 1

   

Sep-Conv:

Sep-Inc:

2. 比较了全3D，全2D，先3D再2D，先2D再3D, 先2D再3D效果最好

R(2+1)D contributions

1. 同样的 先空间+时间 分解卷积，分解之后，Loss下降更快，下降速度val上比train上更快
2. R是Residual

P3D contributions

1. 同样的空间时间分解，比较了下面三种结构：

P3D-A效率最高

---

所有这些方法中，特征加强的越多（光流，idt），最终准确度越高

所有这些模型都是在 Kinetics 和 Sport-1M这种大库上训练，UCF101 HMDB51上这种小库上fintune

CSN contributions

1. 分组卷积，通道上进行组合，大大减少计算量

2. channel separation 扮演了 正则化的作用，得到更低的test error 避免过拟合
3. interaction preserved CSN：先全连接，再depth wise
4. interaction-reduced CSN：直接depth wise