Sculpting dynamical systems for models of neural computation and memory

Dynamical systems are now a mainstay of computational neuroscience. Persistent activity in biological neural networks has been posited to result from dynamical attractors in neural state-space [1], and computation with attractors underlies a variety of models for information processing and memory function in the brain [2–4]. Nevertheless, there exists no general, substrate-free method for the construction of these dynamical systems [2], substrate-free meaning not tied to a particular neural instantiation.

We introduce such a method in this work, with a focus on applications to neural computation and memory modeling. We call this method state-space sculpting. The systems it constructs, or sculpts, are realized as ordinary differential equations. Sculpting is therefore substrate free; it is general in the sense that sculpted systems may contain any number of attractors embedded at any locations in the state-space and the basins of those attractors may be shaped with any smooth geometry. We conjecture that the sculpting process is analogous to learning by synaptic modification in some biological and artificial neural networks.

The method works as follows. First, a set of attractor locations and types (fixed point, periodic, strange) is defined. Then a set of corresponding vector fields is identified: Each of these has one of the defined attractor types at one of the defined locations in its state-space. Next, the basins of these attractors are delineated by defining smooth boundaries for the corresponding vector fields. Finally, the set of vector fields and boundaries is unified into a single smooth dynamical equation using regularization [5].

We have sculpted various 2-dimensional systems by this method, and their dynamics were confirmed in simulation to exhibit the defined attractors and basins. These systems contained arbitrary combinations of fixed point and periodic attractors, as well as repelling regions, with arbitrary basin geometries. Systems of saddle equilibria were also sculpted to demonstrate the versatility of the method. These can be made to exhibit heteroclinic channels and cycles, which are key features of the Winnerless Competition Principle of neural computation [4].

As a first test of our conjecture on the analogy between sculpting and synaptic modification, the well known Hopfield network [6] was analyzed and characterized in sculpting terms. This analysis demonstrated most notably that the network's basins of attraction are delineated by combinations of codimension-1 hyperplanes containing saddle equilibria. The analogue of state-space sculpting in the Hopfield network is therefore limited, since attractor basins cannot be shaped with any smooth geometry.

Moving forward, it is necessary to study more complicated networks to show that they can be cast in the sculpting framework, as well as to extend the sculpting method to higher-dimensional state-spaces and manifolds.