https://www.toutiao.com/a6676762194882855428/

迁移学习时间序列分类

题目：

Transfer learning for time series classification

作者：

Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, Pierre-Alain Muller

来源：

Accepted at IEEE International Conference on Big Data 2018

Machine Learning (cs.LG)

Submitted on 5 Nov 2018

文档链接：

arXiv:1811.01533

代码链接：

https://github.com/hfawaz/bigdata18

摘要

深度神经网络的迁移学习是先在源数据集上训练基网络，然后将学习到的特征(网络的权重)转移到目标数据集上训练的第二个网络的过程。在图像识别和目标定位等计算机视觉任务中，该方法提高了深度神经网络的泛化能力。除了这些应用，深度卷积神经网络(CNNs)最近在时间序列分类(TSC)领域也得到了广泛的应用。然而，与图像识别问题不同的是，在TSC任务中，迁移学习技术还没有得到深入的研究。这是令人惊讶的，因为如果从预先训练的神经网络对模型进行微调，而不是从零开始训练，TSC的深度学习模型的准确性可能会得到提高。本文通过研究如何在TSC任务中传输深度CNNs来填补这一空白。为了评估转移学习的潜力，我们使用UCR存档进行了广泛的实验，UCR存档是最大的公开可用的TSC基准测试，包含85个数据集。对于存档中的每个数据集，我们预先训练一个模型，然后在其他数据集上对其进行微调，得到7140个不同的深度神经网络。这些实验表明，迁移学习可以根据用于迁移的数据集改进或降低模型的预测。因此，为了预测给定目标数据集的最佳源数据集，我们提出了一种基于动态时间扭曲的数据集相似性度量方法。我们描述了我们的方法如何引导转换来选择最佳的源数据集，从而提高85个数据集中71个数据集的准确性。

英文原文

Transfer learning for deep neural networks is the process of first training a base network on a source dataset, and then transferring the learned features (the network's weights) to a second network to be trained on a target dataset. This idea has been shown to improve deep neural network's generalization capabilities in many computer vision tasks such as image recognition and object localization. Apart from these applications, deep Convolutional Neural Networks (CNNs) have also recently gained popularity in the Time Series Classification (TSC) community. However, unlike for image recognition problems, transfer learning techniques have not yet been investigated thoroughly for the TSC task. This is surprising as the accuracy of deep learning models for TSC could potentially be improved if the model is fine-tuned from a pre-trained neural network instead of training it from scratch. In this paper, we fill this gap by investigating how to transfer deep CNNs for the TSC task. To evaluate the potential of transfer learning, we performed extensive experiments using the UCR archive which is the largest publicly available TSC benchmark containing 85 datasets. For each dataset in the archive, we pre-trained a model and then fine-tuned it on the other datasets resulting in 7140 different deep neural networks. These experiments revealed that transfer learning can improve or degrade the model's predictions depending on the dataset used for transfer. Therefore, in an effort to predict the best source dataset for a given target dataset, we propose a new method relying on Dynamic Time Warping to measure inter-datasets similarities. We describe how our method can guide the transfer to choose the best source dataset leading to an improvement in accuracy on 71 out of 85 datasets.

要点

我们的研究结果应该促使大数据从业者在对时间序列进行分类时，不再从零开始训练模型，而是对预先训练好的模型进行微调。特别是，如果设计得当，CNNs可以适应不同长度的时间序列数据集。在我们未来的工作中，我们的目标是再次减少深度神经网络的过拟合现象，通过使用加权DTW重心平均法[48]生成合成数据，因为后者的距离在指导一个复杂的深度学习工具，如转移学习方面给出了令人鼓舞的结果。最后，随着大数据存储库变得越来越频繁，利用现有的与感兴趣的目标数据集类似但并不完全相同的源数据集，使得转移学习方法成为一种吸引人的方法。

图1:有无以电器为源，OSULeaf为目标数据集的转移学习方法的模型损失演化(训练和测试)。(彩色效果最佳)。