研究背景： 在過去的五年裡，人工智慧的發展已經用來作為解決許多複雜問題的方案，例如物體識別和圍棋。然而與人類大腦相比，這些深度神經網路模型需要的訓練量和數據遠遠要多一個數量級。在許多情況下，人類甚至可以從一個例子中學習。洞悉人類的學習方式可以幫助我們開發更有效的算法來訓練人工神經網路。傳感器和物聯網技術的進步使得在實驗期間構建用於數據重新編碼的智能玩具成為可能，並且已有許多相關發表的研究。感覺運動功能在嬰兒早期發展，可以使用嵌入玩具中的傳感器進行量化。要了解兒童的發展，需要進行縱向的研究。然而，大多數縱向研究都集中在語言發展上。
研究目標： 我們的目標是為嬰兒的發展和學習提供新的見解。我們相信，通過長期縱向研究，量化每天正常玩耍玩具時的感知和運動技能可以獲得新的見解。為實現這一目標，我們在此展示了兩種用於嬰兒數據採集之原型玩具之開發。
研究方法： 我們的智能鼓和智能邦哥鼓原是商業玩具，安裝了嵌入式電子設備，用於數據記錄，互聯網訪問和通過聊天App Telegram 的雙向通信。我們的智能安撫奶嘴是一款飛利浦Soothie 安撫奶嘴，配有帶藍牙的慣性測量感測器（IMU），可實時跟踪運動。我們在智能安撫奶嘴中實施並評估了慣性導航系統（INS），以估計運動軌跡。
研究結果： 我們製造，展示和測試了智能小鼓/邦哥鼓和智能安撫奶嘴。我們的聊天機器人將每次按下按鈕轉換為圖像並發送到任何授權用戶的聊天室，以及訂閱戶可以給予鼓/邦哥鼓中的光刺激的命令。我們在智能奶嘴中實施了Madgwick 過濾器和無損卡爾曼濾波器作為用於跟踪相對位置的慣性導航系統。智能安撫奶嘴的軌跡估計誤差對於我們要求的微小運動來說太大，但是軌跡還是可以用於識別運動模式。
研究結論： 我們報告智能玩具之設計和首次現場測試。這些工具顯示了在兒童家中長期縱向數據採集的潛力，但仍存在可靠度和電池壽命方面的挑戰。

Background: The revolution in Artificial intelligence has resulted, within the past five years, in super human solution of many problems, such as object recognition and Go play.
These deep neural network models require more than an order of magnitude more energy and data for training compared to the human brain. Human can in many cases even learn from a single example. New insights into how humans learn could help us develop more efficient algorithms for training artificial neural networks. Advances in sensors and Internet of Things technology has made it possible to build smart toys for data recoding during experiments and a number of such studies have been published. The sensorimotor function develops early in infants and it can be quantified using sensors embedded in toys. To understand the development of children, longitudinal studies are needed. However, most longitudinal studies are focused on language development.
Aim: We aimed to provide new insights into the development and learning of infants. We believe that new insights could be gained through long-term longitudinal studies with daily quantification of the perception and motor skills during normal play with toys. Towards this aim we here present the development of two prototype toys for data collection from infants.
Method: Our smart drum and bongo are commercial toys updated with embedded electronics for data recording, Internet access, and two-way communication through the chat App Telegram. Our smart pacifier is a Philips Soothie pacifier updated with an inertial measurement unit (IMU) with Bluetooth for tracking of movement in real-time. We implemented and evaluated an inertial navigation system (INS) in the smart pacifier to estimate the trajectory of movement.
Results: We built, demonstrated, and tested the smart bongo/drum and smart pacifier.
Our chatbot translates each press of a button to an image sent to the Telegram chat App of any authorized subscriber and commands given by the subscriber into light stimuli in the drum/bongo. We implemented a Madgwick filter and an Unscented Kalman filter in the smart pacifier as the INS for tracking of relative position. The error of the trajectory estimation from smart pacifier is too large for the position estimates to be of use, but the trajectory can be used to recognize the pattern of the movement.
Conclusion: We report on the design and first field tests. These tools show potential for long-term longitudinal data collection in the home of children, but challenges with reliability and battery life remain.

References

Anzulewicz, A., Sobota, K., and Delafield-Butt, J. T. (2016). Toward the autism motor signature: Gesture patterns during smart tablet gameplay identify children with autism. Scientific reports, 6:31107.
Ayub, S., Bahraminasab, A., and Honary, B. (2012). A sensor fusion method for smart phone orientation estimation.
Castro, P., Chiu, P., Kremenek, T., and Muntz, R. (2001). A probabilistic room location service for wireless networked environments. In International conference on ubiquitous computing, pages 18–34. Springer.
Chen, Z., Zhu, Q., and Soh, Y. C. (2016). Smartphone inertial sensor-based indoor localization and tracking with ibeacon corrections. IEEE Transactions on Industrial Informatics, 12(4):1540–1549.
Craig Ramey (1972). The Carolina Abecedarian Project: A Longitudinal and Multidisciplinary Approach to the Prevention of Developmental Retardation.
Cramer, R.-M., Scholtz, R. A., and Win, M. Z. (2002). Evaluation of an ultra-wide-band propagation channel. IEEE Transactions on Antennas and Propagation, 50(5):561–570.
Demetzou, K., Böck, L., and Hanteer, O. (2018). Smart bears don’t talk to strangers: Analysing privacy concerns and technical solutions in smart toys for children. In Living in the Internet of Things: Cybersecurity of the IoT - 2018, pages 1–7.
Dourou, E., Komessariou, A., Riga, V., and Lavidas, K. (2017). Assessment of gross and fine motor skills in preschool children using the peabody developmental motor scales instrument. Eur Psychomotricity J, 9:89–113.
Ekin, C. C., Cagiltay, K., and Karasu, N. (2018). Effectiveness of smart toy applications in teaching children with intellectual disability. Journal of Systems Architecture, 89(February): 41–48.
Gibbs, S. (2015). Hackers can hijack wi-fi hello barbie to spy on your children. The Guardian.
Gorostiza, E. M., Galilea, L., Luis, J., Meca Meca, F. J., Salido Monzú, D., Espinosa Zapata, F., and Pallarés Puerto, L. (2011). Infrared sensor system for mobile-robot positioning in intelligent spaces. Sensors, 11(5):5416–5438.
Gośliński, J., Nowicki, M., and Skrzypczyński, P. (2015). Performance comparison of ekfbased algorithms for orientation estimation on android platform. IEEE Sensors Journal, 15(7):3781–3792.
Goula-Dimitriou, M. and Dasygenis, M. (2016). Teddy bear upgraded with an embedded system to react on feelings. In 2016 5th International Conference on Modern Circuits and Systems Technologies (MOCAST), pages 1–4. IEEE.
Grammer, J. K., Coffman, J. L., Ornstein, P. A., and Morrison, F. J. (2013). Change over time: Conducting longitudinal studies of children’s cognitive development. Journal of Cognition and Development, 14(4):515–528.
Grierson, M. and Kiefer, C. (2013). Noisebear: a wireless malleable multiparametric controller for use in assistive technology contexts. In CHI’13 Extended Abstracts on Human Factors in Computing Systems, pages 2923–2926. ACM.
Hashem, I. A. T., Yaqoob, I., Anuar, N. B., Mokhtar, S., Gani, A., and Ullah Khan, S. (2015). The rise of ”big data” on cloud computing: Review and open research issues. Information Systems, 47:98–115.
Hung, T. and Suh, Y. (2013). Inertial sensor-based two feet motion tracking for gait analysis. Sensors, 13(5):5614–5629. Hunt, T. (2015). When children are breached–inside the massive VTech hack. Retrieved Dec.
Jimenez, A. R., Seco, F., Prieto, C., and Guevara, J. (2009). A comparison of pedestrian deadreckoning algorithms using a low-cost mems imu. In 2009 IEEE International Symposium on Intelligent Signal Processing, pages 37–42. IEEE.
Kaemarungsi, K. and Krishnamurthy, P. (2004). Modeling of indoor positioning systems based on location fingerprinting. In Ieee Infocom 2004, volume 2, pages 1012–1022. IEEE.
Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. Journal of basic Engineering, 82(1):35–45.
Kara, N., Aydin, C. C., and Cagiltay, K. (2013). Investigating the activities of children toward a smart storytelling toy. Educational Technology and Society, 16(1):28–43.
Kieliba, P., Veltink, P. H., Baldi, T. L., Prattichizzo, D., Santaera, G., Bicchi, A., Bianchi, M., and Van Beijnum, B.-J. F. (2018). Comparison of three hand pose reconstruction algorithms using inertial and magnetic measurement units. In 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids), pages 1–9. IEEE.
Kirk, E., Howlett, N., Pine, K. J., and Fletcher, B. (2013). To sign or not to sign? the impact of encouraging infants to gesture on infant language and maternal mind-mindedness. Child Development, 84(2):574–590.
Kirkham, N. Z., Slemmer, J. A., and Johnson, S. P. (2002). Visual statistical learning in infancy: Evidence for a domain general learning mechanism. Cognition, 83(2):B35–B42.
Kluz, R. and Trzepieciński, T. (2014). The repeatability positioning analysis of the industrial robot arm. Assembly Automation, 34(3):285–295.
Kraft, E. (2003). A quaternion-based unscented kalman filter for orientation tracking. In Proceedings of the Sixth International Conference of Information Fusion, volume 1, pages 47–54.
Lampe, M. and Hinske, S. (2007a). Integrating interactive learning experiences into augmented toy environments. In Pervasive Learning Workshop at the Pervasive Conference, May, pages 13–16. Citeseer.
Lampe, M. and Hinske, S. (2007b). Integrating Interactive Learning Experiences into Augmented Toy Environments. Wortkshop on Pervasive Learning, pages 1–9.
Li, Z., Dehaene, W., and Gielen, G. (2007). System design for ultra-low-power uwb-based indoor localization. In 2007 IEEE International Conference on Ultra-Wideband, pages 580–585. IEEE.
Liem, M. C. and Gavrila, D. M. (2014). Coupled person orientation estimation and appearance modeling using spherical harmonics. Image and Vision Computing, 32(10):728–738.
Ljung, L. (1979). Asymptotic behavior of the extended kalman filter as a parameter estimator for linear systems. IEEE Transactions on Automatic Control, 24(1):36–50.
Logan, S., Robinson, L. E., Wilson, A., and Lucas, W. (2012). Getting the fundamentals of movement: a meta-analysis of the effectiveness of motor skill interventions in children. Child: care, health and development, 38(3):305–315.
Luinge, H. J. and Veltink, P. H. (2004). Inclination measurement of human movement using a 3-d accelerometer with autocalibration. IEEE Transactions on neural systems and rehabilitation engineering, 12(1):112–121.
Madgwick, S. O., Harrison, A. J., and Vaidyanathan, R. (2011). Estimation of imu and marg orientation using a gradient descent algorithm. In 2011 IEEE international conference on rehabilitation robotics, pages 1–7. IEEE.
Marti, P. (2010). Bringing playfulness to disabilities. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries, pages 851–856. ACM.
Marti, P., Moderini, C., Giusti, L., and Pollini, A. (2009). A robotic toy for children with special needs: From requirements to design. In 2009 IEEE International Conference on Rehabilitation Robotics, pages 918–923.
Martín Ruiz, M. L., Valero Duboy, M. Á., Linden, M., Nunez Nagy, S., and Gutiérrez García, Á. (2015). Foundations of a smart toy development for the early detection of motoric impairments at childhood. International Journal of Pediatric Research, 1(2):1–5.
Medina, C., Segura, J., and De la Torre, A. (2013). Ultrasound indoor positioning system based on a low-power wireless sensor network providing sub-centimeter accuracy. Sensors, 13(3):3501–3526.
Medina, C., Segura, J. C., and Holm, S. (2012). Feasibility of ultrasound positioning based on signal strength. In 2012 International Conference on Indoor Positioning and Indoor Navigation (IPIN), pages 1–9. IEEE.
Moye, D. (2015). Talking Doll Cayla Hacked To Spew Filthy Things. The Huffington Post.
Nogueira, S. F., Figueiredo, E. M., Gonçalves, R. V., and Mancini, M. C. (2015). Relation between hand function and gross motor function in full term infants aged 4 to 8 months. Brazilian journal of physical therapy, 19(1):52–60.
Piaget, J. (1964). Part I: Cognitive development in children: Piaget development and learning. Journal of research in science teaching, 2.
Piper, M. C., Darrah, J., Maguire, T. O., and Redfern, L. (1994). Motor assessment of the developing infant. Saunders Philadelphia.
Plowman, L. and Luckin, R. (2004). Interactivity, interfaces, and smart toys. Computer, 37(2):98–100.
Raffle, H. (2006). Kinesthetic media: touch, toys & interactive materials. In ACM SIGGRAPH 2006 Educators program, page 8. ACM.
Ren, H. and Kazanzides, P. (2012). Investigation of attitude tracking using an integrated inertial and magnetic navigation system for hand-held surgical instruments. IEEE/ASME Transactions on Mechatronics, 17(2):210–217.
Rhemtulla, M. and Tucker-Drob, E. M. (2011). Correlated longitudinal changes across linguistic, achievement, and psychomotor domains in early childhood: Evidence for a global dimension of development. Developmental Science, 14(5):1245–1254.
Rivera, D., García, A., Alarcos, B., Velasco, J., Ortega, J., and Martínez-Yelmo, I. (2016a). Smart toys designed for detecting developmental delays. Sensors, 16(11):1953.
Rivera, D., García, A., Alarcos, B., Velasco, J. R., Ortega, J. E., and Martínez-Yelmo, I. (2016b). Smart Toys Designed for Detecting Developmental Delays. Sensors (Basel, Switzerland), 16(11).
Roy, D. (2009). New horizons in the study of child language acquisition. Keynote at Interspeech. Runge, A., Baunach, M., and Kolla, R. (2011). Precise self-calibration of ultrasound based indoor localization systems. In 2011 International Conference on Indoor Positioning and Indoor Navigation, pages 1–8. IEEE.
Sabatini, A. M. (2011). Estimating three-dimensional orientation of human body parts by inertial/magnetic sensing. Sensors, 11(2):1489–1525.
Sénéchal, M. and Lefevre, J.-a. (2002). Parental Involvement in the Development of Children’s Reading Skill : A Five-Year Longitudinal Study. Child development, 73(2):445–460.
Smith, A., Balakrishnan, H., Goraczko, M., and Priyantha, N. (2004). Tracking moving devices with the cricket location system. In Proceedings of the 2nd international conference on Mobile systems, applications, and services, pages 190–202. ACM.
Stallings, L. M. (1973). Motor skills: development and learning. WC Brown Company.
Thelen, E. (2000). Grounded in the world: Developmental origins of the embodied mind. Infancy, 1(1):3–28.
Thompson, R. H., Cotnoir-Bichelman, N. M., McKerchar, P. M., Tate, T. L., and Dancho,
K. A. (2007). Enhancing early communication through infant sign training. Journal of
applied behavior analysis, 40(1):15–23.
Valenti, R. G., Dryanovski, I., and Xiao, J. (2015). A linear kalman filter for marg orientation
estimation using the algebraic quaternion algorithm. IEEE Transactions on Instrumentation
and Measurement, 65(2):467–481.
Vega-Barbas, M., Pau, I., Ferreira, J., Lebis, E., and Seoane, F. (2015). Utilizing smart
textiles-enabled sensorized toy and playful interactions for assessment of psychomotor
development on children. Journal of Sensors, 2015.
Wan, E. A. and Van Der Merwe, R. (2000). The unscented kalman filter for nonlinear estimation.
In Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications,
and Control Symposium (Cat. No. 00EX373), pages 153–158. Ieee.
Weinberg, H. (2002). Using the adxl202 in pedometer and personal navigation applications.
Analog Devices AN-602 application note, 2(2):1–6.
Westeyn, T. L., Abowd, G. D., Starner, T. E., Johnson, J. M., Presti, P. W., and Weaver, K. A.
(2012). Monitoring children’s developmental progress using augmented toys and activity
recognition. Personal and Ubiquitous Computing, 16(2):169–191.
Xiao, J., Liu, Z., Yang, Y., Liu, D., and Han, X. (2011). Comparison and analysis of indoor
wireless positioning techniques. In 2011 International conference on computer science
and service system (CSSS), pages 293–296. IEEE.
Yan, H., Shan, Q., and Furukawa, Y. (2018). Ridi: Robust imu double integration. In Proceedings
of the European Conference on Computer Vision (ECCV), pages 621–636.
Yilmaz, R. M. (2016). Educational magic toys developed with augmented reality technology
for early childhood education. Computers in Human Behavior, 54:240–248.
Yuan, X., Yu, S., Zhang, S., Wang, G., and Liu, S. (2015). Quaternion-based unscented
kalman filter for accurate indoor heading estimation using wearable multi-sensor system.
Sensors, 15(5):10872–10890.
Zaman, B., Van Mechelen, M., and Bleumers, L. (2018). When Toys Come to Life: Considering
the Internet of Toys from an Animistic Design Perspective. Proceedings of the 17th
ACM Conference on Interaction Design and Children - IDC ’18, pages 170–180.