论文题目：Jointly Non-Sampling Learning for Knowledge Graph Enhanced Recommendation

• 摘要

Knowledge graph (KG) contains well-structured external information and has shown to be effective for high-quality recommendation. However, existing KG enhanced recommendation methods have largely focused on exploring advanced neural network architectures to better investigate the structural information of KG. While for model learning, these methods mainly rely on Negative Sampling (NS) to optimize the models for both KG embedding task and recommendation task. Since NS is not robust (e.g., sampling a small fraction of negative instances may lose lots of useful information), it is reasonable to argue that these methods are insufficient to capture collaborative information among users, items, and entities.
知识图(KG)包含结构良好的外部信息，已被证明是高质量推荐的有效方法。然而，现有的KG增强推荐方法主要集中在探索先进的神经网络架构，以更好地研究KG的结构信息。而对于模型学习，这些方法主要依靠负抽样(NS)对KG嵌入任务和推荐任务的模型进行优化。由于NS不是健壮的(例如，采样一小部分负面实例可能会丢失大量有用的信息)，有理由认为这些方法不足以捕获用户、项目和实体之间的协作信息。

In this paper, we propose a novel Jointly Non-Sampling learning model for Knowledge graph enhanced Recommendation (JNSKR). Specifically, we first design a new efficient NS optimization algorithm for knowledge graph embedding learning. The subgraphs are then encoded by the proposed attentive neural network to better characterize user preference over items. Through novel designs of memorization strategies and joint learning framework, JNSKR not only models the fine-grained connections among users, items, and entities, but also efficiently learns model parameters from the whole training data (including all non-observed data) with a rather low time complexity. Experimental results on two public benchmarks show that JNSKR significantly outperforms the state-of-the-art methods like RippleNet and KGAT. Remarkably, JNSKR also shows significant advantages in training efficiency (about 20 times faster than KGAT), which makes it more applicable to real-world largescale systems.
本文提出了一种基于联合非抽样学习的知识图增强推荐模型(JNSKR)。具体来说，我们首先设计了一种新的高效的知识图嵌入学习的NS优化算法。子图然后由所提出的注意神经网络编码，以更好地表征用户对项目的偏好。JNSKR通过新颖的记忆策略和联合学习框架设计，不仅对用户、项目、实体之间的细粒度连接进行建模，还能以较低的时间复杂度从整个训练数据(包括所有未观察到的数据)中高效地学习模型参数。两个公共基准测试的实验结果表明，JNSKR显著优于最先进的方法，如RippleNet和KGAT。值得注意的是，JNSKR在训练效率上也显示出显著优势(大约比KGAT快20倍)，这使得它更适用于现实世界的大规模系统。

• Non-sampling Learning for Top-K Recommendation

For implicit data, the observed interactions are rather limited, and non-observed examples are of a much larger scale. To learn from such a sparse data, there are generally two optimization strategies: 1) negative sampling strategy [5, 14, 25] and 2) non-sampling (whole-data based) strategy [7, 15, 16]. The first strategy samples a fraction of negative instances from non-observed entries, while the second one sees all the non-observed data as negative. In previous work (especially neural recommendation studies), negative sampling is widely adopted for efficient training. However, some recent studies have shown that sampling would inevitably limit the recommendation performance as it can ignore some important examples, or lead to insufficient training of them [7, 15, 43, 46]. In contrast, non-sampling strategy leverages the whole data with a potentially better coverage, but inefficiency can be an issue [7]. Some efforts have been devoted to resolving the inefficiency issue of non-sampling learning. For instance, Pilaszy et al. [24] describe an approximate solution of Alternating Least Squares (ALS). He et al. [15] propose an efficient ALS with non-uniform missing data. Some researchers [43, 46] study fast Batch Gradient Descent (BGD) methods. Recently, Chen et al. [7, 8] derive a flexible non-sampling loss for neural recommendation models, which achieves both effective and efficient performance
对于隐式数据，观测到的交互作用是相当有限的，而未观测到的例子的规模要大得多。对于这样的稀疏数据，一般有两种优化策略:1)负采样策略[5,14,25]和2)非采样(基于全数据)策略[7,15,16]。第一种策略从未观察到的条目中抽取一部分负实例，而第二种策略将所有未观察到的数据视为负数据。在以往的工作中(特别是神经推荐研究)，负采样被广泛地用于有效的训练。然而，最近的一些研究表明，采样不可避免地会限制推荐性能，因为采样会忽略一些重要的例子，或导致对这些例子的训练不足[7,15,43,46]。相反，非抽样策略利用整个数据，具有更好的潜在覆盖率，但效率低下可能是一个问题。在解决非抽样学习效率低下的问题上，已经做了一些努力。例如，Pilaszy等人的[24]描述了交替最小二乘(ALS)的近似解。他等人提出了一种具有非均匀缺失数据的有效ALS。一些研究者[43,46]研究了快速批量梯度下降(BGD)方法。最近，Chen等人[7,8]为神经推荐模型推导了一种灵活的非采样损失，它实现了有效和高效的性能。

尽管现有的非抽样研究取得了成功，但它们主要集中在CF方法上，只考虑了用户与物品之间的两要素关系。将这些方法直接应用于学习由实体-关系-实体三联体组成的KG增强推荐具有重要意义。据我们所知，这是第一次研究KG增强推荐的有效非抽样方法。

• 高效的非采样协同过滤
  最近，一些研究发现，非抽样策略对实现最优推荐性能非常有帮助[7,15,16,46]。我们简要介绍了高效的非抽样协同过滤方法，它是为学习用户对项目的偏好而设计的。对于隐式数据，一种常用的非采样损失是最小化用户反馈yuv与预测结果yuv之间的差异[16]：
  
• Efficient Non-sampling Knowledge Graph Embedding

知识图嵌入是在保持图结构的前提下，将实体和关系转换为向量表示的一种有效方法。它已被广泛应用于知识增强推荐算法中[1,34,38]。现有的知识图嵌入方法[2,21,44]主要利用负抽样方法进行模型优化，但近年来的研究表明其不具有鲁棒性[7,43]。本文提出将非抽样策略应用于知识图嵌入学习。其中，对于一批实体B，图嵌入学习的平方损失定义为

另外，非观测数据的损失可以用所有数据的损失与正数据的损失之间的残差来表示。我们有下面的推导：