在機器學習的時間序列數據中，雖然容易收集大量無標籤數據，但標籤數據仍然難以獲得。因此，如何使用未標籤數據增強模型學習逐漸受到關注。本研究關注於深度多任務學習，特別是硬參數共享方法，當任務間關聯性不強時，共享所有參數可能會導致學習干擾，稱為負轉移。基於此，我們提出一個新框架，將訓練分為學習和分享兩階段，讓模型決定共享知識的多寡，動態的進行參數共享。此外，提出一個新損失函數，使任務之間的神經元功能性對齊。實驗結果顯示，即使任務間關聯性低或不確定，我們的方法也能有效避免負轉移。

In the realm of time-series data within machine learning, while there exists an ease in amassing substantial volumes of unlabeled data, procuring labeled datasets remains challenging. This has underscored the rising interest in leveraging unlabeled data to bolster model learning. The current research emphasizes deep multi-task learning, especially the hard parameter sharing approach. Notably, when inter-task correlations are not robust, sharing all parameters could result in learning interference, known as negative transfer. In light of this, we propose a novel framework, bifurcating the training into 'learning' and 'sharing' phases, allowing the model to determine the extent of knowledge sharing and dynamically engaging in parameter sharing. Furthermore, we introduce a new loss function to align the functionalities of neurons between tasks. Experimental outcomes reveal that even in situations of low or ambiguous inter-task correlations, our approach can effectively circumvent negative transfer.

[1] D. Miorandi, S. Sicari, F. D. Pellegrini, and I. Chlamtac, “Internet of things: Vision, applications and research challenges,” Ad Hoc Networks, vol. 10, pp. 1497–1516, 2012.
[2] “Google nest cam,” http://www.tp-ontrol.hu/index.php/TP Toolbox.
[3] “Provigial video analytics,” https://pro-vigil.com/.
[4] “Cyanconnode smart electricity metering,” https://cyanconnode.com/applications/ smart-electricity-metering/.
[5] “Analog device: Industrial automation.” https://www.analog.com/en/applications/ markets/industrial-automationtechnology-pavilion-home.html.
[6] J. Torriti, M. A. G. Hassan, and M. Leach, “Demand response experience in europe: Policies, programmes and implementation,” Energy, vol. 35, pp. 1575–1583, 2010.
[7] “Institute for electric innovation 2022 annual report,” https://www.edisonfoundation.net/ -/media/Files/IEI/about/IEI 2022 Annual-Report.pdf.
[8] C. Shortall, Una; Esser, “Annual report on the results of monitoring the internal electricity and natural gas markets in 2021,” https://www.ceer.eu/ documents/104400/7517827/ACER-CEER+Annual+Report+on+the+Results+ of+Monitoring+the+Internal+Electricity+and+Natural+Gas+Markets+in+2021/ 44783ab3-410c-1560-319f-664677d934e0.
[9] M. Kaselimi, E. E. Protopapadakis, A. Voulodimos, N. D. Doulamis, and A. D. Doulamis, “Towards trustworthy energy disaggregation: A review of challenges, methods, and perspectives for non-intrusive load monitoring,” Sensors (Basel, Switzerland), vol. 22, 2022.
[10] L. Pereira, D. Costa, and M. Ribeiro, “A residential labeled dataset for smart meter data analytics,” Scientific Data, vol. 9, 2022.
[11] Y. Wang, Q. Chen, T. Hong, and C. Kang, “Review of smart meter data analytics: Applications, methodologies, and challenges,” IEEE Transactions on Smart Grid, vol. 10, pp. 3125–3148, 2018.
[12] M. A. Devlin and B. P. Hayes, “Non-intrusive load monitoring and classification of activities of daily living using residential smart meter data,” IEEE Transactions on Consumer Electronics, vol. 65, pp. 339–348, 2019.
[13] B. V¨olker, M. Pfeifer, P. M. Scholl, and B. Becker, “A framework to generate and label datasets for non-intrusive load monitoring,” Energies, 2020.
[14] N. Noury, K.-A. Quach, M. Berenguer, H. Teyssier, M.-J. Bouzid, L. Goldstein, and M. Giordani, “Remote follow up of health through the monitoring of electrical activities on the residential power line - preliminary results of an experimentation,” 2009 11th International Conference on e-Health Networking, Applications and Services (Healthcom), pp. 9–13, 2009.
[15] H. I. Fawaz, G. Forestier, J. Weber, L. Idoumghar, and P.-A. Muller, “Deep learning for time series classification: a review,” Data Mining and Knowledge Discovery, vol. 33, pp. 917–963, 2018.
[16] L. Chen, D. Chen, F. Yang, and J. Sun, “A deep multi-task representation learning method for time series classification and retrieval,” Inf. Sci., vol. 555, pp. 17–32, 2021.
[17] F. Karim, S. Majumdar, H. Darabi, and S. Chen, “Lstm fully convolutional networks for time series classification,” IEEE Access, vol. 6, pp. 1662–1669, 2017.
[18] K. D. Anderson, A. Ocneanu, D. R. Carlson, A. G. Rowe, and M. E. Berges, “Blued : A fully labeled public dataset for event-based non-intrusive load monitoring research,” 2012.
[19] H. Wang, G. Henri, C.-W. Tan, and R. Rajagopal, “Activity detection and modeling using smart meter data: Concept and case studies,” 2020 IEEE Power & Energy Society General Meeting (PESGM), pp. 1–5, 2020.
[20] H. K. Iqbal, F. H. Malik, A. Muhammad, M. A. Qureshi, M. N. Abbasi, and A. R. Chishti, “A critical review of state-of-the-art non-intrusive load monitoring datasets,” Electric Power Systems Research, p. 106921, 2020.
[21] M. D. Wagy, J. C. Bongard, J. P. Bagrow, and P. D. H. Hines, “Crowdsourcing predictors of residential electric energy usage,” IEEE Systems Journal, vol. 12, pp. 3151–3160, 2017.
[22] Y. Chen, B. Hu, E. J. Keogh, and G. E. A. P. A. Batista, “Dtw-d: time series semisupervised learning from a single example,” Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, 2013.
[23] H. Wang, Q. Zhang, J. Wu, S. Pan, and Y. Chen, “Time series feature learning with labeled and unlabeled data,” Pattern Recognit., vol. 89, pp. 55–66, 2019.
[24] L. Jing and Y. Tian, “Self-supervised visual feature learning with deep neural networks: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, pp. 4037–4058, 2019.
[25] S. Jawed, J. Grabocka, and L. Schmidt-Thieme, “Self-supervised learning for semisupervised time series classification,” Advances in Knowledge Discovery and Data Mining, vol. 12084, pp. 499 – 511, 2020.
[26] R. Caruana, “Multitask learning,” Machine Learning, vol. 28, pp. 41–75, 1997.
[27] L. Xi, Z. Yun, H. Liu, R. Wang, X. Huang, and H. Fan, “Semi-supervised time series classification model with self-supervised learning,” Eng. Appl. Artif. Intell., vol. 116, p. 105331, 2022.
[28] X. Liu, F. Zhang, H. Liu, and H. Fan, “itimes: Investigating semisupervised time series classification via irregular time sampling,” IEEE Transactions on Industrial Informatics, vol. 19, pp. 6930–6938, 2023.
[29] H. Choi and P. Kang, “Multi-task self-supervised time-series representation learning,” ArXiv, vol. abs/2303.01034, 2023.
[30] Y. Zhang and Q. Yang, “A survey on multi-task learning,” IEEE Transactions on Knowledge and Data Engineering, vol. 34, pp. 5586–5609, 2022.
[31] J. Yu and J. Jiang, “Learning sentence embeddings with auxiliary tasks for cross-domain sentiment classification,” in Conference on Empirical Methods in Natural Language Processing, 2016.
[32] R. Caruana, “Multitask learning: A knowledge-based source of inductive bias,” in International Conference on Machine Learning, 1993.
[33] S. Ruder, “An overview of multi-task learning in deep neural networks,” ArXiv, vol. abs/1706.05098, 2017.
[34] G. W. Hart, “Nonintrusive appliance load monitoring,” Proc. IEEE, vol. 80, pp. 1870–1891, 1992.
[35] K. He, L. Stankovi´c, J. Liao, and V. Stankovi´c, “Non-intrusive load disaggregation using graph signal processing,” IEEE Transactions on Smart Grid, vol. 9, pp. 1739–1747, 2018.
[36] Y. Lin and M.-S. Tsai, “Non-intrusive load monitoring by novel neuro-fuzzy classification considering uncertainties,” IEEE Transactions on Smart Grid, vol. 5, pp. 2376–2384, 2014.
[37] G. Elafoudi, L. Stankovi´c, and V. Stankovi´c, “Power disaggregation of domestic smart meter readings using dynamic time warping,” 2014 6th International Symposium on Communications, Control and Signal Processing (ISCCSP), pp. 36–39, 2014.
[38] J. Kelly and W. J. Knottenbelt, “Neural nilm: Deep neural networks applied to energy disaggregation,” Proceedings of the 2nd ACM International Conference on Embedded Systems for Energy-Efficient Built Environments, 2015.
[39] J. Kim, T.-T.-H. Le, and H. Kim, “Nonintrusive load monitoring based on advanced deep learning and novel signature,” Computational Intelligence and Neuroscience, vol. 2017, 2017.
[40] D. Murray, L. Stankovi´c, V. Stankovi´c, S. Lulic, and S. Sladojevic, “Transferability of neural network approaches for low-rate energy disaggregation,” ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8330– 8334, 2019.
[41] Y. Pan, K. Liu, Z. Shen, X. Cai, and Z. Jia, “Sequence-to-subsequence learning with conditional gan for power disaggregation,” ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3202–3206, 2020.
[42] A. Faustine, L. Pereira, H. Bousbiat, and S. Kulkarni, “Unet-nilm: A deep neural network for multi-tasks appliances state detection and power estimation in nilm,” Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring, 2020.
[43] D. Murray, L. Stankovi´c, and V. Stankovi´c, “Explainable nilm networks,” Proceedings of the 5th International Workshop on Non-Intrusive Load Monitoring, 2020.
[44] H. Fan, F. Zhang, R. Wang, X. Huang, and Z. Li, “Semi-supervised time series classification by temporal relation prediction,” ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3545–3549, 2021.
[45] A. Rasmus, M. Berglund, M. Honkala, H. Valpola, and T. Raiko, “Semi-supervised learning with ladder networks,” in NIPS, 2015.
[46] D.-H. Lee, “Pseudo-label : The simple and efficient semi-supervised learning method for deep neural networks,” 2013.
[47] M. N. Rizve, K. Duarte, Y. S. Rawat, and M. Shah, “In defense of pseudo-labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning,” ArXiv, vol. abs/2101.06329, 2021.
[48] S. Laine and T. Aila, “Temporal ensembling for semi-supervised learning,” ArXiv, vol. abs/1610.02242, 2016.
[49] D. Berthelot, N. Carlini, I. J. Goodfellow, N. Papernot, A. Oliver, and C. Raffel, “Mixmatch: A holistic approach to semi-supervised learning,” ArXiv, vol. abs/1905.02249, 2019.
[50] K. Sohn, D. Berthelot, C.-L. Li, Z. Zhang, N. Carlini, E. D. Cubuk, A. Kurakin, H. Zhang, and C. Raffel, “Fixmatch: Simplifying semi-supervised learning with consistency and confidence,” ArXiv, vol. abs/2001.07685, 2020.
[51] G. E. Hinton, S. Osindero, and Y. W. Teh, “A fast learning algorithm for deep belief nets,” Neural Computation, vol. 18, pp. 1527–1554, 2006.
[52] Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle, “Greedy layer-wise training of deep networks,” in NIPS, 2006.
[53] D. Rao, F. Visin, A. A. Rusu, Y. W. Teh, R. Pascanu, and R. Hadsell, “Continual unsupervised representation learning,” in Neural Information Processing Systems, 2019.
[54] M. Schwarzer, A. Anand, R. Goel, R. D. Hjelm, A. C. Courville, and P. Bachman, “Dataefficient reinforcement learning with self-predictive representations,” in International Conference on Learning Representations, 2020.
[55] J. Baxter, “A bayesian/information theoretic model of learning to learn via multiple task sampling,” Machine Learning, vol. 28, pp. 7–39, 1997.
[56] S. M. Tabatabaei, S. Dick, and W. Xu, “Toward non-intrusive load monitoring via multilabel classification,” IEEE Transactions on Smart Grid, vol. 8, pp. 26–40, 2017.
[57] S. Gidaris, P. Singh, and N. Komodakis, “Unsupervised representation learning by predicting image rotations,” ArXiv, vol. abs/1803.07728, 2018.