It is necessary to choose a visual aid that is appropriate for the material and audience.
Many machine learning algorithms, especially neural networks that are so popular now, borrow ideas for their effective work from the knowledge of neurophysiologists about how our brain works. It turns out that among the vast amount of information about genes, neurons, synapses and neurotransmitters, one can single out quite specific principles of how our brain learns and makes decisions. These principles include hierarchy, feedback, layer-by-layer learning, lateral suppression, etc. To combine these ideas and implement them in a new algorithm for controlling a technical device (as they say, to build a neuromorphic algorithm) is the task of this project.
One of the key technologies that we use in this direction is the hierarchical temporary memory framework - HTM. This developing area is not yet adapted to dynamical systems and does not support the idea of reinforcement learning. Expanding HTM in this direction is one of the key topics.
Hawkins J., Ahmad S., Cui Y. A Theory of How Columns in the Neocortex Enable Learning the Structure of the World // Frontiers in Neural Circuits. 2017. Vol. 11. P. 1–18. LINK
Nugamanov E., Panov A.I. Hierarchical Temporal Memory with Reinforcement Learning // Procedia Computer Science. 2020. Vol. 169. P. 123–131. ScienceDirect
Gradskov A.A., Panov A.I. Capsule graphs of computations in the reinforcement learning problem // Eighth International Conference "System Analysis and Information Technologies" CAIT - 2019 (July 8 - 14, 2019, Irkutsk - Listvyanka, Russia): Proceedings of the conference. Moscow: FRITs IU RAN, 2019, pp. 207–211.
Daylidyonok I., Frolenkova A., Panov A.I. Extended Hierarchical Temporal Memory for Motion Anomaly Detection // Biologically Inspired Cognitive Architectures 2018. BICA 2018. Advances in Intelligent Systems and Computing / ed. Samsonovich A. V. Springer, 2019. Vol. 848. P. 69–81. Springer
George D., Hawkins J. Towards a mathematical theory of cortical micro-circuits // PLoS Comput. Biol. 2009. Vol. 5, № 10. P. e1000532.