Introduction
Carver Mead coined the term neuromorphic engineering (NE) in the late 1980s. It explains how to recreate the neurobiological structures and computational capacities found in the human nervous system using very large-scale integration (VLSI) systems, which incorporate software and digital, analog, and mixed-mode VLSI (1). Unlike the Von Neumann digital machine, the human brain can operate in an analog environment and do complicated tasks in huge quantities with little power and space needs (2). To comprehend NE, one can employ hardware elements like threshold switches, oxide-based memristors, transistors, and spintronic memory.
The design concepts and structural frameworks of many Northeastern research fields, such as physics, biology, computer science, mathematics, material science, and electronic engineering, are based on human neurological systems (3). The rest of the paper is organized in this manner. Section “Motivation”explains why the NE survey was conducted. The many models used in engineering computers are covered in Section “Representation of the various models”. Section “Different training algorithms” gives a summary of the various learning strategies and techniques used. The hardware implementations are discussed in Section “Hardware implementation”. The most recent NE initiatives are covered in Section “Current initiatives”. Section “Applications” discusses the kind of applications where NE has proven useful in earlier studies. A list of open research questions and a forward-looking assessment of NE are included in the study’s conclusion.
Motivation
We reviewed the research in this area because of the growing trend over time in favor of the studies and advancements in NE. The goal of computer scientists has been to simulate organic neural networks in virtual machines. This effort has led to significant breakthroughs in the fields of machine learning, artificial intelligence, and artificial neural networks (ANNs). This study focuses on the development of neuromorphic computing and the analysis of various neuron models. We examine a few of the motivations behind the development of neuromorphic systems across time, drawing on the sources mentioned in the literature.
Representation of the various models
Neuron models
Neurons generate electrical impulses in response to external inputs, which they subsequently transmit to other neurons (4). Some of the most important physical traits and physiological roles of a neuron are listed below.
Axons and dendrites
Axons communicate with other neurons by carrying the inputs that dendrites get from other cells (4, 5). Through synaptic connections, a neuron in the dendritic tree can receive inputs from a variety of cell types.
Ion channels
Certain proteins called ion channels generate electrical signals that go through the heart, brain, and muscles (6). The numerous ion channels that permit ions to enter and exit cells include those that cross the membranes for potassium (K+), sodium (Na+), calcium (Ca2+), and chloride (Cl–) ions (4). Ion channels, which react to a shift in potential across the cell membrane by opening and closing, regulate ion passage.
Membrane potential
According to Daroff’s 2014 encyclopedia, the membrane potential is the difference in voltage between a neuron’s inner and outer. A neuron’s resting potential, or the voltage across its cell membrane about its surroundings, is roughly -70 mV (4). The concentration gradient is maintained by the ion channels in the cell membrane, which maintain a steady membrane potential difference. While sodium ions are far more concentrated in a neuron’s extracellular fluid, potassium ions are substantially more prevalent inside a neuron than outside.
Hyperpolarization and depolarization
One phase in the hyperpolarization process is to make the membrane potential more negative (4). On the other hand, depolarization results in either positive or less negative membrane potential values.
Action potential
Neurons generate a special type of electrical signal that travels along their long axons. These signals are also called action potentials (7) and “spikes” or “impulses” (8). The action potential opens ion channels at synapses, allowing calcium (Ca2+) ions to enter and releasing a chemical messenger called a neurotransmitter.
Refractory period
The short time after an action potential when the axon becomes resistant to further stimulation is known as the refractory period (9). It is identified by an increase in the threshold for the firing of the next pulse, which follows a firing pulse. The moment after the spike is created when the system can no longer be aroused is known as this (10). Absolute and relative refractory periods are the two types of refractoriness. The amount of time that a second spike cannot be unquestionably generated, regardless of stimulus intensity, is known as the absolute refractory period. The relative refractory period, which follows the absolute refractory period, is defined by suppressing the initiation of a second spike rather than its impossibility.
The following discusses several neuron models.
Hodgkin-Huxley model
The conductance of Na+ and K+ of the axon membrane are varied in this model to systematically represent the ionic currents in the giant squid axon (11). A mathematical model is described in Koslow and Subramaniam (12) that uses many voltage clamp tests to show the time- and voltage-dependent characteristics of Na+ and K+ conductances. The Hodgkin-Huxley model (13), a system of differential equations derived from this discovery, established the ionic foundation of the action potential.
Izhikevich model
The bursting and spiking behaviors of certain cortical neurons are explained by this model (14), which combines the computational efficiency of the integrate-and-fire neuron with the dynamics of the Hodgkin-Huxley model. According to the study Van Schaik et al. (15), this model is constructed in the first-order log domain using two translinear multipliers and low-pass filters. The gadget could also change the input current, bias voltages, and currents to display the spike patterns of the neuron model.
Mihalas-Niebur neuron
The generalized integrate-and-fire neuron model is another name for this model (16). Several of the bursting and spiking features observed in actual biological neurons are also found in this model (15, 17). It explains each of the state variables using simple first-order differential equations, unlike earlier simplified Hodgkin-Huxley neuron models (18). By Folowosele et al. (19), the Mihalas-Niebur model is implemented in complementary metal–oxide–semiconductor (CMOS). Among the spiking characteristics discussed here are rebound spiking, which generates spikes after being released from a hyperpolarized state, hyperpolarized spiking, which generates spike trains in response to continuous stimuli, and phasic spiking, which produces a single spike at the beginning of a stimulus but remains inactive for the remainder of the stimulus.
Leaky-integrate-and-fire model (LIF)
The Leaky-integrate-and-fire (LIF) model, which describes how action potentials are generated in response to continuous inputs, is generalized by the spike response model (20). The membrane potential differential equations were used to build the spike response model, which was mostly based on filters (21), which examines a straightforward model that faithfully mimics the spiking behavior of neurons. As previously mentioned, LIF models are frequently employed to mimic neural activation.
For discrete-time input spiking The LIF model’s equation can be expressed as
where vrest is the cell’s resting potential, I(t) is the injected current, τ is the membrane’s “leaky” time constant, and R is the membrane’s resistance. R is assumed to be unity in spiking neural network (SNN) applications, and
xi(t) is the equivalent spiking input to the ith presynaptic neuron, and wi is the synaptic weight between the target neuron and the presynaptic neuron.
The LIF model is an upgraded integrate and fire model that accounts for membrane voltage leak, a feature of real neurons that results in an increase in the interspike interval (ISI) over time for a normal spike train.
The membrane potential experimentally determined and the matching stimulus current during sweep 51 of a single epileptic cell ID (cell id: 488392806) are shown in Figure 1. Sweeps are a collection of experiments on the electrophysiological data of cells.
Figure 1. Typical excitation (current applied) and the corresponding response (membrane potential) of a single neuron (Epileptic Cell id – 488392806, Sweep – 51) (22).
The response and the ISI between two consecutive spikes for a particular neuron are represented in Figure 2.
Figure 2. Neuronal Response for a single neuron (22).
Adaptive exponential integrate-and-fire-model
The membrane potential and adaptation current can be used to mathematically express the two-dimensional model (20, 23). It is an extension of the exponential LIF neuron, which can reproduce the upswing of an action potential with an exponential function and its downswing with a reset condition. When the membrane potential approaches the threshold voltage in this case, the exponential term causes the voltage to increase rapidly (24). Furthermore, subthreshold adaptation and spike-triggered adaptation are controlled by a few model parameters.
McCulloch-Pitts model
According to Chakraverty et al. (25) and Mehlig (26), this model is the first neural network model with weighted directed paths connecting neurons. A generalized model of the McCulloch-Pitts neuron can be used to determine the threshold in an axon with time-dependent nonlinear dynamics (27). The weighted inputs from the other neurons in the network are added linearly by this model to determine the value of a binary unit (28).
Synapse models
The junction of two neurons where communication occurs is called a synapse. Information is transmitted from a presynaptic source neuron to a postsynaptic target neuron (29). The human body contains two different types of synapses: electrical and chemical (30). Electrical synapses allow ions and chemicals to connect directly to neurons. A chemical synapse, on the other hand, delays the signal’s delivery from the presynaptic neuron to the target cell. The postsynaptic cell needs a neurotransmitter that diffuses across synapses and binds to receptors to generate an electrical signal. In a chemical synapse, the space between the target and presynaptic source membranes is known as the synaptic cleft (31). When the neurotransmitter binds to the receptor, the ion channels open or close. The target cell’s membrane potential changes as a result of this. Positively charged ions (Na+) may enter a postsynaptic cell and depolarize it when the membrane voltage changes, which increases the possibility that the action potential will spread. An excitatory postsynaptic potential is the term used to describe this occurrence (32). In contrast, inhibitory postsynaptic potentials (IPSPs) are caused by the outflow of positive ions (K+) or the influx of negative ions (Cl–) from the postsynaptic cell. These ISPs lead to hyperpolarization and decrease the likelihood of action potential propagation (33).
The many types of synaptic contact are then discussed. Presynaptic neurons join postsynaptic neurons through their dendritic spines to form axodendritic synapses (34). Axosomatic synapses usually terminate in the soma, or cell body, of the postsynaptic neuron. Axons from presynaptic neurons and postsynaptic neurons can form synaptic connections known as axo-axonic synapses.
The strength or weakness of synaptic connections over time is referred to as synaptic plasticity (35). There are two types of synaptic plasticity: short-term plasticity (STP) and long-term plasticity (LTP). Short-term depression (STD) and short-term facilitation (STF) are the two types of STPs. A few milliseconds to several minutes is the average duration of an STP. The postsynaptic response (STD) drops and the synaptic strength (STF) increases as presynaptic activity repeats (36). LTP is an activity-dependent synaptic strength shift that can last anywhere from a few minutes to several hours or more (37). Like STP, there are two types of LTP: long-term depression (LTD, decreased synaptic strength) and long-term potentiation (LTPo, increased synaptic strength). Spike-timing-dependent plasticity (STDP) (38, 39) is a phenomenon in which the amount and sign of LTD and LTPo can be determined by the order and time interval between the presynaptic and postsynaptic spikes.
The many synaptic models are covered in the section below.
Synapse models based on non-conductance
Current dependence characterizes these models (40). The several non-conductance-based synapse models are as follows:
Current source synapse model. This hypothesis states that an action potential activates a voltage-controlled current source transistor that is linked in series with a switching transistor (31, 41). This small model has no current dynamics. The postsynaptic cell is not significantly impacted by the input pulses’ speed. When VLSI systems are connected to pulse-based neural networks, the neural code is generated using average firing rates (42). This circuit may be able to match currents in addition to doing two-quadrant multiplications (43).
Reset-discharge synapse model. Comprising three P-channel field effect transistors (pFETs) and one capacitor, this synaptic circuit generates an excitatory postsynaptic current (EPSC) that decays exponentially with time and lasts longer than the input pulse duration (31). The non-linear summing of all input spikes prevents the creation of the linear summation of postsynaptic currents, which is frequently required in synaptic models (41).
Linear charge-discharge synapse. At both times, the circuit allows the output current to increase and decrease. All transistors are assumed to be saturated and to operate below the threshold (31) and (41), respectively. This model’s high-impedance node and non-linear integrator characteristics are among its disadvantages. A low input frequency reduces the node’s ability to encode the frequency by lowering the voltage across the node.
Current mirror integrator synapse. A current mirror integrator circuit is constructed in this model by joining a diode-connected transistor with another transistor (31). The capacitor integrates the charge, whereas the diode-connected transistor at the input side leaks it away (44). This circuit features a non-linear saturation value, the maximum amplitude of which is dictated by the weight and time constant bias, and the output current rises as the input firing rates do. This circuit works when it is in a stable condition.
Log-domain synapse. In this model, the sub-threshold MOSFET’s gate-to-source voltage and channel current are logarithmic (31). The circuit may be susceptible to the trans-linear principle due to its exponential relationship (45, 46). According to this theory, the product of the currents flowing in clockwise and anticlockwise directions is identical for a group of transistors coupled in a closed-loop configuration (47).
Log-domain integrator synapse. Only an input spike triggers the activation of this model’s input current (31). Unlike all of the previous models discussed above, this circuit has linear filtering qualities, which means that all incoming input spikes are summed up linearly. Local adaptation techniques, which can resolve chip-area problems that occur during the manufacture of VLSI multi-neuron chips, are not used by the synapses in this instance. These circuits can help implement synaptic dynamics in neuromorphic networks (48).
Differential pair integrator synapse. As a log-domain filter, this circuit can generate the exponential dynamics observed in the excitatory and inhibitory postsynaptic currents at synapses (49). The differential pair integrator (DPI) circuit described in the publication (50) works with both homeostatic synaptic scaling and spike-based learning principles. A software control system with a loop is used to implement a synapse circuit in a VLSI chip in order to stabilize the properties of homeostatic control. Homeostasis is a self-regulating process by which biological systems try to maintain stability by adapting to the conditions that are favorable for survival (51). By overcoming challenges that arise when manufacturing big charge packets that are delivered into the neuron’s integrating capacitor for brief input spikes, this form of circuit offers the advantage of perhaps avoiding the need for additional pulse-extender circuits (31).
Conductance-based synapse circuit models
These are voltage-dependent models (40). Various conductance-based based synapse models are discussed below:
Single transistor synapse. A transistor functioning in the ohmic region can generate postsynaptic current based on the difference between the reversal potential and the membrane potential (31). This paradigm includes the ability to learn, compute, and store non-volatile information (52). It is possible to perform long-term weight storage, calculate the product of the input and the weight value, and update the weight value in a single-transistor synapse using the backpropagation (BP) learning algorithm. Their low power consumption and tiny size facilitate the development of dense synaptic arrays. In this instance, the increment-decrement functions are proportional to the source current’s power.
Switched-Capacitor model. Several types of synaptic conductances, including α-amino-3-hydroxy-5-methylisoxazole-4-propionic acid (AMPA), gamma-aminobutyric acid (GABA), and N-methyl-D-asparate (NMDA), may be accurately represented by this conductance-based synapse model (53). This idea suggests that it is easier to switch between the technologies. This article describes a mixed-signal neuromorphic system that calculates the conductance value of many synapses using digital circuitry and switched-capacitor synapses. The digital logic can be synthesized, but the analog building blocks are common. Some STP presynapses and stop-learning synapses are implemented via the SC circuits in a neuromorphic system that uses the CMOS process, as explained in Noack et al. (54). This circuit allows for real-time operations by minimizing leakage effects.
Synchronous-Asynchronous release STP model
The entering action potential is exactly matched by the neurotransmitter’s release. Asynchronous neurotransmitters are released through this stochastic synchronized release mechanism, known as synchronous-asynchronous release, which can intensify even when there is no spiking activity (55). By utilizing experimental data from the inhibitory fast-spiking synapses of human epileptic cells, Neff et al. (56) showed that this model produces the behavior of an asynchronous neurotransmitter release. Both synchronous and asynchronous neurotransmitter release affect the synaptic regulation pattern of a single neuron. This model generates the behavior of an asynchronous neurotransmitter release, as demonstrated by Neff et al. (56) using experimental data from the inhibitory fast-spiking synapses of human epileptic cells. An individual neuron’s pattern of synaptic control is influenced by both synchronous and asynchronous neurotransmitter release.
Multi memristive synapses model
According to Ansari (57), a memristor is a device that has two terminals that connect the linkage between electric charge and magnetic flux. In this case, the resistance depends on the polarity and magnitude of the applied voltage as well as the temporal length of the potential (58). These gadgets can operate similarly to the physical principles of a brain as well as synapses. According to this paradigm, the equivalent conductance of several devices determines the synaptic weight (59). The resolution and dynamic range of the synapses are increased by using multiple devices. This type has a high crossbar compatibility and is very reliable. In order to complete the recognition of different letters, a memristor crossbar circuit is discussed in Paper (60). Because the H two-terminal memristor has better properties than most physical memristors, it was chosen. In this model, a memristor powers the winner-take-all and mirror circuits. The performance of the circuit has been seen in a number of memristive distributions and variations.
Hybrid CMOS/Memristor synapse model
The hybrid CMOS synapse model (61) consists of a high density of ultra-thin sheets of resistance switching devices coupled with a CMOS subsystem that is reliable, flexible, and very effective. Based on crystallization in chalcogenide materials and the impact of magneto-resistance in a magnetic tunnel junction, the memristor is made up of three devices: resistive switching memory (RRAM), spin transfer torque memory (STTRAM), and phase change memory (PCM) (62). Memristors offer great capacity, reduced cell sizes, and quick speed. With this method, neuromorphic networks with high density, functionality, and low power consumption can be created using RRAM and PCM. Utilizing a DPI circuit, the input spikes are integrated, and the circuit in Fayyazi et al. (63) shows how to build many arrays of synaptic weight elements with independent inputs.
Network models
These models discuss the connections and interactions of different neurons and synapses. The various network models are discussed below.
Recurrent neural networks
A recurrent neural network (RNN) has at least one feedback loop to help with activation (64, 65). This accelerates the temporal processing and sequence learning of the networks and enables tasks such as temporal prediction and sequence recognition. The most popular kind of RNN architecture is a Multi-Layer Perceptron (MLP) with a few more loops added to the standard design. The results of a CMOS single-chip implementation and an analog VLSI system with a straightforward and scalable architecture for learning in dynamical RNNs are covered in Cauwenberghs (66). Instead of computing the gradient using an explicit model of the network dynamics, the learning architecture uses a stochastic approach that directly examines how the network error depends on the parameters. An essential part of intrusion detection, a key element of information security, is the capacity to precisely identify different kinds of network invasions. A deep learning-based intrusion detection system is modeled in this paper (67), which also assesses the model’s performance in binary and multiclass classification, as well as the effects of different learning rates and the number of neurons on the suggested model’s performance. The study employs a deep learning method for intrusion detection called recurrent neural networks (RNN-IDS).
Stochastic neural networks
These neural networks are created by randomly altering the network by giving its neurons stochastic weights or stochastic transfer functions (68). This can help with problem optimization since random oscillations allow it to escape from local minima. The on-off transition of an oxide synaptic device assumes a probabilistic character under weak programming conditions (69). This switching variability can be used to build a stochastic learning rule. Similar system performance was found for the binary synapse, which uses stochastic learning, and the analog synapse, which uses depression-only learning, in an examination of a winner-take-all network simulation for the orientation classification function.
Distributed multichip networks
A grid network for broadcasting the spikes by relaying them from chip to chip, in a multichip neuromorphic system, has been implemented in Merolla et al. (70). Unlike a bus, the capacity of the grid does not decrease when chips are added to it, so the grid can be expanded. Also, the cycle time of the grid is shorter than the bus, and hence latency is not increased in multiple relays. The addresses of the chip for indicating the source of spikes are automatically assigned by the implementation of an asynchronous relay. In the paper Shao et al. (71), the importance of multichip modules having fine-grained chipsets used in areas with large computation and on-chip storage has been discussed. Here, a 36-chipset prototype of the multichip module system for deep learning inference, termed Simba, has been built and implemented.
Spiking neural networks
Third-generation neural network models that mimic the information flow in actual neurons are called SNNs (72). Using computationally realistic cycle time neurons, this network aims to close the gap between neuroscience and machine learning (73). The incidence of spikes, which signify a number of biological activities, including the membrane potential of the neuron, may be determined using differential equations. SNNs work with discrete events instead of continuous value chipsets because they employ spikes. Reaching its potential causes a neuron to spike and then reset. Because the SNN can encode the temporal information in signals, it outperforms the non-spiking ones in terms of power. Vreeken’s (74) “spiking,” the author of Ghosh-Dastidar and Adeli (75), presents the multi-SNN and its training technique, which involves information being transferred from one neuron to another via many synapses in the form of multiple spikes. The performance of the model and its learning procedure are computed using the three pattern recognition problems. These consist of the eXclusive OR (XOR) problem (76), the Fisher iris plant classification problem (77), and the electroencephalogram (EEG) epilepsy and seizure detection problem (78).
Deep neural networks
An ANN having several layers between the input and output layers is called a deep neural network (DNN). Despite their wide range of configurations, neural networks all have the same basic elements: synapses, neurons, weights, biases, and functions. Technology, particularly the quick development of brain imaging technologies, has always been crucial to the study and emphasis of new perspectives on the structure and function of the brain. Medical research makes extensive use of image processing methods to enhance early diagnosis and treatment. Thus far, DNNs have shown exceptional performance in applications involving classification and segmentation. Consequently, a deep wavelet autoencoder (DWA) method for image compression is presented in this paper (79). DWA combines the basic feature reduction capabilities of an autoencoder with the picture decomposition capabilities of a wavelet transform. The combination of both greatly minimizes the size of the feature set for a longer-term DNN classification assignment. The suggested DWA-DNN image classifier was taken into consideration after a collection of brain images was collected. In recent years, recommendation systems have been widely used to provide customers with suggestions across a range of commercial platforms. One of the primary algorithms utilized in recommendation systems is collaborative filtering. Despite the simplicity and efficiency of these methods, the sparsity of the data and the scalability of the approach limit their performance, and it is challenging to improve the quality of the proposed findings. As a result, a model that integrates deep learning technology with a collaborative filtering recommended approach is proposed (80).
Different training algorithms
The various learning algorithms used in a neuromorphic system are discussed below.
Unsupervised learning
A machine accepts inputs in unsupervised learning, but it does not receive the desired outputs or rewards from its surroundings (81). It can be used to identify patterns in data that go beyond what would be expected from pure unstructured noise. The study Kreiser et al. (82) discusses spike-based unsupervised learning in a neuromorphic system that is constructed with analog electronics that use spiking neuron circuits and low-power synapse circuits with on-chip plasticity. The winner-take-all network configuration of the silicon neuron populations, in this case, allows the system to automatically learn how to categorize the input patterns of the various spike rates.
Supervised learning
In supervised learning, one can observe both the input and the output of a component (83). Here, the learning element is provided almost the precise value of a particular input function and even tries to change the representation of the function to match the information in the feedback. By promoting synaptic plasticity on the single-spike system, this learning algorithm, as proposed in paper (84), enhances learning ability and attains nearly identical accuracy when compared to similarly configured SNNs and ANNs. In order to modify the slope of the activation function (AF) associated with the neurons, Panda and Panda (85) have devised new BP learning rules. The combined rules for linking sigmoid function slopes and weights are then exposed to the ANN structure in order to facilitate faster training. The trained ANN also solves the benchmark problems of nonlinear system identification and classification. The nonlinear AF in a typical artificial neural model follows the weight sum method. Panda and Panda (86) suggest a modified artificial neural network (MANN) and determine the optimal BP learning method by prioritizing the AF above the connecting weights of each artificial neuron. The asymmetric decoding method may successfully generate synaptic weight categorization and updating when paired with supervised learning. A spike-based supervised learning approach for the handwritten digit recognition problem and network optimization for the number of learning parameters for its implementation on memory and energy-constrained hardware are provided in the paper (87). Four modified radial basis function neural network (RBFNN) models have been presented in Panda and Panda (88). These four modified RBFNN models have a number of popular uses.
These days, image processing is essential for the early detection and management of a number of illnesses, such as cancers of the breast, lung, and brain. Nowadays, the eye examination method is an effective way to diagnose most cancers. Due to differences between and within observers, the visual evaluation of small biopsy images by humans is incredibly subjective, time-consuming, and contradictory. This will allow for the early identification of the cancer and its makeup for the purpose of final treatment and repair. This machine learning-based backpropagation neural network (MLBPNN) (89) brain tumor classification technique reduces the inter-observer variability while assisting pathologists in identifying threats more precisely and effectively.
Reinforcement learning
Reinforcement learning is a closed-loop algorithm where the learning system’s actions influence subsequent inputs. Instead of knowing what has to be done in advance, the learner experiments to determine which action will yield the greatest reward (90). This learning is almost the same as supervised learning, with the exception that it requires a critic instead of a teacher (91). The inverted pendulum technique (92, 93), has been used to create a memristive-based neuromorphic circuit for reinforcement learning in Wu et al. (94). Here, the learning algorithms significantly lower the number of weight updates.
Evolutionary algorithms
Evolutionary algorithms, also known as stochastic direct search methods based on population, mimic natural evolution (95). Evolutionary algorithms optimize the neural network’s synaptic connection based on a task-dependent fitness function (95). Unlike the popular machine learning methods, these algorithms allow both the network architecture and the change of link weights.
Hardware implementation
The hardware implementation of a neuromorphic circuit is discussed below.
Digital
Using CMOS technology, the learning mechanism present in biological neurons has been applied in Seo and Seok (96). Here, transistor scaling and dynamic voltage scaling have been used to implement dense and low-power hardware using memory circuits and digital logic for neuromorphic processor design. Additionally, two neuromorphic processors have been implemented that can integrate and fire synapses and neurons for pattern recognition and clustering applications. A ring-oscillator-based synapse and neuro module in a CMOS neuromorphic conductance-based 5 nm technology with higher time is described in a study (97). Without biasing, the digital neuron module generates frequent, rapid spikes and intrinsic bursts that resemble biological firing patterns. Excitatory and inhibitory synapses are both components of the synapse module. Paper, Kim et al. (98), describes the architecture of an optimized SNN digital hardware accelerator that uses an efficient synapse memory structure. This design lowers the hardware resources needed to keep the network size and performance in balance. The presynaptic weight scaling can be used to decrease the size of synaptic memory. The neuron machine, a specialized hardware design for modeling large-scale neuromorphic systems, is described in Ahn (99). This machine system consists of one digital hardware neuron, a memory unit, and a large-scale fine-grained pipelined circuit. The inherent parallelism of neural networks can be used by a neuron machine on a big scale since pipelining and large-scale memories are widely used.
Analog
Spiking dynamics of neurons in a network of analog silicon neurons with dynamic conductance-based synapses were demonstrated in the journal (100). The analog VLSI processor reproduces the continuous-time dynamics of a digital programmable network of four biophysical neurons. Mill et al. (101) discuss a model of synaptic depression-based stimulus-specific adaptation and how it is used in neuromorphic analog VLSI. Biologically appropriate spike trains that resemble the stimuli used in physiological studies have been used to evaluate the hardware device. An analog circuit design and component implementation for an adaptive neuromorphic olfaction chip were provided in Koickal et al. (102). A DC offset cancellation circuit is part of the chemical sensor’s signal processing circuitry. To carry out the odor information, the data from the on-chip odor sensors is decoded using an analog circuit and employed as temporal spiking signals.
Current initiatives
The recent trends in neuromorphic hardware are discussed below.
SpiNNaker
SpiNNaker, a digital neuromorphic architecture, was developed by the University of Manchester to simulate biologically inspired SNNs (103, 104). At the heart of it all is a low-power general-purpose ARM processor that can use basic ARM instructions to translate all of the neural model equations directly into the machine. The complete system consists of 57,000 nodes with 1,036,800 ARM9 processors and seven terabytes of RAM. Each node features 128 MB of off-die synchronous dynamic RAM and roughly 18 cores (105). SpiNNaker’s massively parallel architecture is powered by a specifically developed million-core system-on-chip (106). The full-scale simulations of a cortical microcircuit on SpiNNaker have been explored in van Albada et al. (107) using a neurobiological timescale. Here, LIF neurons make up the cortical microcircuit. SpiNNaker’s performance is compared to that of neural network simulation software (NEST) in terms of accuracy, power consumption, and runtime. A visual recognition classification system implemented on an FPGA and a spiking classifier running on SpiNNaker have been demonstrated in Tapiador-Morales et al. (108) using a spiking convolution processor. This processor convolves the visual input stimulus from a card image. After the convolution, the event is fired regardless of which one reaches a threshold value. The SNN model in SpiNNaker is used to process the output events, and the different card symbols—club, spade, heart, and diamond—are categorized. The movement of a neuromorphic audio-guided hexapod robot driven by spiking central pattern generators is described in Gutierrez-Galan et al. (109). The many robot gaits generated by these pattern generators are implemented on a SpiNNaker board. The spikes generated by these generators are then transmitted to the servomotors using pulse-width modulation signals generated by the FPGA.
In Mayr et al. (110), the author describes the path from the 130 nm CMOS SpiNNaker, a machine with one million cores, to the 22 nm fully depleted silicon on insulator (FDSOI) SpiNNaker2, a system-on-chip with ten million cores. The simulation capability of SpiNNaker2 is more than fifty times larger than that of its predecessor. Millisecond-latency sensor processing, high-throughput machine learning, rigorous real-time operation, and excellent energy efficiency are some of SpiNNaker2’s capabilities. The Deep Rewiring algorithm has been used to build a DNN with learning capabilities on a neuromorphic system, according to the publication (111). The Deep Rewiring training method maintains sparse connections by continuously rewiring the network during the production process. The SpiNNaker2 has continued to run during the installation’s training and testing stages.
Loihi from intel
The Loihi, an entirely digital, asynchronous neuromorphic processor, was released by Intel (112). SNNs may be trained on-chip thanks to its microcode programmable learning engine. Numerous target platforms, including Loihi silicon, an FPGA emulator, and a software functional simulator, are part of the Loihi toolchain. A compiler and runtime library are also included for creating and running SNNs on Loihi. SNNs are specified using the Application Programming Interface, which is based on Python. This device implements SNNs using 128 custom cores with 1,024 neurons each (113). Numerous characteristics for the fields, including dendritic compartments, synaptic delays, hierarchical connectivity, and programmable synaptic learning rules, may be included thanks to Loihi (114). It can compute a variety of optimization problems faster than conventional solvers and simulate a convolution process based on spikes.
Several Loihi-based neuromorphic technologies developed by Intel are listed in Table 1. The USB-stick-form-size device Kapoho Bay, which allows a direct address-event representation interface to event-based sensors like a camera, is composed of one or two Loihi chips (115). At first, Wolf Mountain provided the researchers with a Loihi-based board that contained roughly four Loihi chips. Intel builds the framework for larger systems using the FPGA expansion board that comprises Nahuku. The Nahuku chip boards used to construct the Pohoiki Beach system contain eight to 32 Loihi chips each. An extension of the Pohoiki Beach system, Pohoiki Springs comprises approximately 24 Nahuku cards (116). With a power consumption of only roughly 300 Watts, Intel’s Pohoiki Springs neuromorphic system can gather the odor with high accuracy and fewer training samples. According to the paper Imam and Cleland (117), an algorithm has been developed and implemented on the Loihi system based on the structure of the mammalian olfactory bulb. This algorithm can quickly perform online learning and aid in identifying the smell from samples when there is noise present. High-dimensional signals enclosed in unknown environmental conditions can likewise be identified using the aforementioned approach.
Table 1. Series of Loihi-based neuromorphic systems.
TrueNorth from IBM
IBM developed TrueNorth, a massively parallel, fully digital neuromorphic system with 256 million synapses and one million neurons (118). TrueNorth’s design can compute, communicate, and store information without a clock signal. It is also fault-tolerant, scalable, configurable, and distributed. In Moran et al. (119), TrueNorth’s neurosynaptic system presented a spinal image segmentation technique utilizing a deep learning network deployed on neuromorphic hardware. This is done by comparing the network’s performance with human-made disk segmentation and spinal vertebral segmentation. To analyze a stream of data and determine what kinds of features it includes, this system employs a sophisticated neural network model.Citation: Nurse2016decoding. Before a TrueNorth chip is designed, a neural network must be created, trained, and validated using an existing dataset. It is then necessary to use the network to determine the chip’s settings. The work Löhr et al. (120) has introduced a technique for mapping the canonical neural circuit model, which aids in describing the experimental data from various computational neuroscience areas.
Human brain project
The human brain project aims to increase our knowledge of the human brain by providing the European brain research infrastructure for neuroscience and neuroinspired research, as well as rigorous data sharing and collaboration (121). The human brain is the foundation for our ability to think, feel, act, remember, be conscious, and be self-aware. This project contributes to the development of a novel understanding of the human brain and explores platforms that can incorporate all the information gathered about the anatomy and physiology of the brain. New models that can be validated by supercomputer simulations are planned using this data (122). Virtual prototypes of neuromorphic, neuroprosthetic, and robotic devices; the construction of virtual laboratories for drug simulation, basic and clinical studies, etc.; and the generation of new neuroscientific data for modeling, research, and development in the domains of supercomputing, informatics, and modeling are also beneficial.
Applications
Communication systems
Neuromorphic VLSI circuits require a communication system’s substructure (123). In addition to tailoring the data of the numerous system components, a very sophisticated network is needed to transmit pulse events among the numerous synapses and neuron circuits. In the article Thanasoulis et al. (124), a packet-based communication network that offers inter-node connectivity and pulse stimulation has been built on an FPGA. This further makes it possible for a host unit to fully design and control a neuromorphic system. This device has a huge playback memory that allows for quick system configuration, monitoring, and stimulation in addition to capturing brain activity and pulse events. An architecture that can handle the routing needs of a multi-node and accelerated neuromorphic system is presented in Thanasoulis et al. (125). Neural events can be routed in a parallel, tree-like fashion to various postsynaptic neuron blocks with an adjustable propagation delay for each transmission thanks to the capacity to repeat pulses at each node. In this instance, network traffic is generally decreased using the link-tag routing method. The design uses less hardware resources while increasing throughput. A study, Tsurumi et al. (126), claims that pulse width modulation neuromorphic circuits are appropriate for SSD-based near-data computing. The neuron in question is made up of a microcontroller, a charge pump, clock counters, and ten pairs of oscillators. This type uses less electricity and produces superior energy efficiency. This circuit is constructed using a common 180 nm CMOS technology, just like the peripheral circuits of NAND flash memory.
Robotics
A mixed-signal neuromorphic circuit of a joint-driving central pattern generator with adjustable oscillation frequency, duty cycle, and phase lag between joints is presented in the study (127). This can be accomplished by altering the circuits’ settings with a few external commands. The core pattern generator, which is powered by a few sensors, adjusts the output pattern in response to sensory inputs. A coordinated multiple-leg controller that replicates the adaptive mobility of a leg-bearing animal can be created by applying the central pattern generator’s module design, which is used here for a single-leg controller, to several similar modules. The study, Mitchell et al. (128), discusses the development of an autonomous robot that avoids obstacles in its environment by using a neuromorphic computing system. Because it is based on SNNs, the neural network’s implementation adaptability is significantly increased.
Image processing
The paper, Subramaniam (129), explores the methods and algorithms utilized in the emerging field of neuromorphics to achieve a number of image processing tasks, including object identification, selective attention, and picture segmentation, among others. In this case, memristive devices have been used to enhance performance on the previously described visual tests. Paper by Scribner et al. (130) demonstrates how to concurrently rebuild a picture with full intensity and high resolution while detecting and identifying the dominant frequency of a quick temporal event utilizing spike-based data from a neuromorphic readout integrated circuit. The +1 or -1 bits of asynchronous spikes that make up this integrated circuit’s output are comparable to the ON and OFF pulses present in a biological retina.
Successfully integrating synthetic systems with the real world while meeting the demands for accuracy, robustness, and flexibility is crucial. A growing number and variety of sensors are being used to do this. For instance, system integration with state-of-the-art technology is both gratifying and challenging due to noise, production variances, and signal swings. By moving from a common amplitude domain to biology-inspired adaptive spike-domain processing, based on the generic self-X notion, a feasible solution is offered. In this work Abd et al. (131) designed and built a neuromorphic concept and the prototype of an adaptive spiking sensory front-end with self-X characteristics using XFAB CMOS 0.35 μm technology.
A serious and growing public health concern is Alzheimer’s disease (AD), a degenerative brain illness. It provides a concise overview of the causes, risk factors, clinical signs, diagnosis, treatment, and ongoing research on AD. It cites Gupta (132) on adversarial methods. The main goals of managing AD are to lessen symptoms and improve the lives of those who are impacted. A case study on AD detection is provided using an AD Detection Network (ADD-NET) implementation and the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset.
Voice recognition task
The automatic speech recognition task entails transforming the acoustics with varying amplitudes of the sound waveforms for mapping to the frequency domain, which is used in feature extraction, as stated in article (133). Here, the relative contributions of acoustic modifications and neuromorphic hardware to the success rate of speech recognition are quantified. The increase in word success rate from the reservoir’s computing device in comparison to the acoustic transformation is also calculated. In the paper Wu et al. (134), SNNs are used to model acoustics. The model’s effectiveness is evaluated on a variety of voice recognition tasks that require a large vocabulary. The outcomes showed that SNNs’ voice recognition performance is almost on par with ANNs with the same network topology.
Robots
Robots are extensively employed and greatly enhance daily life and industry. Recently, there has been a lot of interest in creating more intelligent robots through a deep confluence of neurology and robotics because of the quick advancements in artificial intelligence and neuromorphic computing. Robot control has been enhanced through the use of memristor-based neuromorphic circuits, which are utilized to construct hardware neural networks (135). These circuits’ features increase robot speed, intelligence, and energy efficiency. Both in the workplace and in daily life, robots are useful because they can speed up industrial production, help with household chores, and perform medical treatments. They are always creating new uses and breaking new ground. According to Gupta (136), only a small proportion of robots are actually intelligent and nimble. Like the human brain, they are unable to make decisions on their own based on sensory or spatiotemporal data. As a result, they find it difficult to carry out procedures and activities in intricate and changing settings with the same accuracy, reliability, and flexibility as people (137, 138). The potential for developing highly sophisticated and human-like robot assistants is considerable, despite the fact that it is frightening.
In Figure 3, memristor-based neuromorphic circuit systems and the human nervous system are compared in detail. Hardware neural networks can be built using neuromorphic circuits based on memristors in four different ways: array topologies, distinct control mechanisms, unique device designs, and analog-digital hybrid platforms. They control the integration of impulses, just like synaptic connections do in the human brain. Memristors or conventional electromechanical actuators can be used to create bio-inspired drive systems that mimic the functions of motor neurons in the human nervous system. Through perception and feedback of the generated outcomes, hardware neural networks may fully regulate robot behavior. This system is genuinely biomimetic.
Figure 3. The analogy between organic brain systems and artificial neural systems made of neuromorphic devices (135).
Natural language processing
The automatic computational processing of human languages through algorithms that produce input that is human-produced and output that appears natural is known as natural language processing. Goldberg (139) is cited. Neural networks provide a powerful and very attractive learning tool for natural language problems. Converting discrete symbols into continuous vectors in a low-dimensional environment is a crucial aspect of language processing in neural networks. When words must be embedded, isolated, and discrete, symbols are transformed into manipulable mathematical objects. It is easier to generalize the behavior of one word to another since the distances between the words and the vectors are almost equal. The network learns to represent words as vectors during training. In the publication (140), the mapping process of RNN on IBM’s
TrueNorth was illustrated using a natural language processing job. Here, a learning methodology has been used to help map RNNs on spiking neuron substrates and is compatible with neuromorphic systems.
Computer vision
The study by Rajasekharan et al. (141) discusses the advantages of neuromorphic circuits on FDSOI technology for computer vision applications, such as color detection, erosion and dilation operations, etc. Here, we suggest using an actuated pan-tilt camera platform to transform static picture data into neuromorphic vision datasets. For instance, the paper, Orchard et al. (142), demonstrates how to automatically convert the existing datasets of static images for computer vision into datasets for neuromorphic vision. This method ensures an approximation of the noise and imperfections derived from the recordings of the environment by using the actual recordings from a neuromorphic sensor. By using the motion of a camera instead of the image on a monitor, it also more accurately detects motion and removes the timing errors caused by display updates.
Object tracking and detection
Using neuromorphic vision with fuzzy processing is one possible way to mimic the visual cortex’s ability to recognize human heads (143). A moving car in a conference room has successfully identified a human in a changing lighting environment, demonstrating the robustness and practicality of neuromorphic vision. The study, Han and Han (144), used the visual cortex’s characteristics to extract features from an image that was taken by a bus camera. The results show that by detecting pedestrians who are closer to the camera and ignoring those who are farther away, it is possible to detect any possible hazard in a car application.
Pattern recognition
An application-specific integrated circuit (ASIC) with integrated and firing neuron circuits and a variety of switched-resistor-based memristors for pattern recognition is described in the article (145). Here, LTspice simulations are run, and a neural network circuit is set up for pattern recognition using memristors. The results showed that four patterns in 3 × 4 pixel input images could be learned and recognized by the circuit. In the paper Jiang et al. (146), a neuromorphic hardware device for pattern recognition is described. The device quickly takes pictures of its environment, which a single-layer neural network then classifies. Postsynaptic neurons and electronic synapses are represented by threshold-controlled neurons and metal-oxide resistive random access memory (RRAM), respectively, to simplify the process and conserve system space.
Conclusion
Real-time interaction between the algorithm and its physical implementation while addressing issues is at the core of NE. When it comes to cognitive processes, this synergy is more complex and intriguing than it is for sensory and motor interactions with the environment. Instead of the primarily feed-forward, reactive neuromorphic systems of the past, NE is now being used to construct adaptive-acting, cognitive neuromorphic systems. Although it has long been a vital cognitive mechanism in the toolkit of NE, selective attention has mostly functioned as a bottom-up process that relies on information and short-term memory.
Particularly for neurons that depend on analog circuits, scalability and the inherent challenges of these circuits present serious challenges. To meet the changing needs of neuromorphic computing, more research might be required to provide solutions that use pre-synaptic spike-driven architecture to consume fewer resources. This can lead to a large reduction in memory access. SNNs may benefit from further computational advantages and energy savings through asynchronous processing. Future studies might look into including asynchronous mechanisms to better align these models with real brain systems.
Consent to publish declaration
Not applicable
Consent to participate declaration
Not applicable
Data availability statement
Data can be available on the Allen Institute for Brain Science website, which is a public website.
Ethics statement
Not applicable
Author contributions
The first author in this paper has an 80% contribution, and the other authors contribute 20% to this research work.
Funding
There is no funding source available for this research work.
Research involving humans and /or animals
It is a review article. Our research work involves human brain neurons.
Informed consent
Permission is usually given by a patient to a physician for treatment after being fully informed of the risks and benefits.
Conflicts of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
1. Calimera A, Macii E, Poncino M. The human brain project and neuromorphic computing. Func Neurol. (2013) 28(3):191.
2. Rajendran B, Sebastian A, Schmuker M, Srinivasa N, Eleftheriou E. Low-power neuromorphic hardware for signal processing applications: a review of architectural and system-level design approaches. IEEE Signal Proc Mag. (2019) 36(6):97–110.
3. McDonnell MD, Boahen K, Ijspeert A, Sejnowski TJ. Engineering intelligent electronic systems based on computational neuroscience [scanning the issue]. Proc IEEE. (2014) 102(5):646–51.
4. Dayan P, Abbott LF. Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems. Cambridge, MA: MIT Press (2001). 806 p.
5. Kandel ER, Schwartz JH, Jessell TM. Nerve cells and behavior. 2nd ed. Principles of Neural Science. New York (NY): Elsevier (1991). p. 24–5.
6. Ackerman MJ, Clapham DE. Ion channels—basic science and clinical disease. N Engl J Med. (1997) 336(22):1575–86.
8. Purves D, Augustine GJ, Fitzpatrick D, Hall W, LaMantia A, McNamara J, et al. Neuroscience. Sunderland, Mass: De Boeck, Sinauer (2014). p. 15–6.
9. Sarpeshkar R, Watts L, Mead C. Refractory Neuron Circuits. CNS Technical Report Number CNS-TR-92-08 (1992).
10. Selmi F, Braive R, Beaudoin G, Sagnes I, Kuszelewicz R, Barbay S. Relative refractory period in an excitable semiconductor laser. Phys Rev Lett. (2014) 112(18):183902.
11. Abbott L, Kepler TB. Model neurons: from Hodgkin-Huxley to hopfield. In: Garrido L editor. Statistical Mechanics of Neural Networks. Springer (1990). p. 5–18.
12. Koslow S, Subramaniam S. Hodgkin-Huxley models. In: Koslow S, Subramaniam S editors. Electrophysiological Models In: Databasing the Brain: From Data to Knowledge. Wiley (2004).
13. Hodgkin AL, Huxley AF. Currents carried by sodium and potassium ions through the membrane of the giant axon of Loligo. J Physiol. (1952) 116(4):449–72.
15. Van Schaik A, Jin C, McEwan A, Hamilton TJ, Mihalas S, Niebur E. A log-domain implementation of the Mihalas-Niebur neuron model. Proceedings of 2010 IEEE International Symposium on Circuits and Systems. IEEE (2010). p. 4249–52.
16. Varghese V, Molin JL, Brandli C, Chen S, Cummings RE. Dynamically reconfigurable silicon array of generalized integrate-and-fire neurons. 2015 IEEE Biomedical Circuits and Systems Conference (BioCAS). IEEE (2015). p. 1–4.
17. Jolivet R, Lewis TJ, Gerstner W. Generalized integrate-and-fire models of neuronal activity approximate spike trains of a detailed model to a high degree of accuracy. J Neurophysiol. (2004) 92(2):959–76.
18. Indiveri G, Linares-Barranco B, Hamilton TJ, Van Schaik A, Etienne-Cummings R, Delbruck T, et al. Neuromorphic silicon neuron circuits. Front Neurosci. (2011) 5:73.
19. Folowosele F, Hamilton TJ, Etienne-Cummings R. Silicon modeling of the Mihalas-Niebur neuron. IEEE Trans Neur Net. (2011) 22(12):1915–27.
20. Gerstner W, Kistler WM, Naud R, Paninski L. Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition. Cambridge: Cambridge University Press (2014).
21. Teeter C, Iyer R, Menon V, Gouwens N, Feng D, Berg J, et al. Generalized leaky integrate-and-fire models classify multiple neuron types. Nat Commun. (2018) 9(1):1–15.
22. Allen Institute for Brain Science.Allen Cell Types Database. Allen Institute for Brain Science (2016). Available online at: http://celltypes.brain-map.org
23. Hertäg L, Hass J, Golovko T, Durstewitz D. An approximation to the adaptive exponential integrate-and-fire neuron model allows fast and predictive fitting to physiological data. Front Comput Neurosci. (2012) 6:62.
24. Brette R, Gerstner W. Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. J Neurophysiol. (2005) 94(5):3637–42.
25. Chakraverty S, Sahoo DM, Mahato NR. Mcculloch-Pitts neural network model. Concepts of Soft Computing. Springer (2019). p. 167–73.
27. Szu H, Rogers G. Generalized Mcculloch-Pitts neuron model with threshold dynamics. Int’l Joint Conf. Neural Networks. IEEE (1992).
28. Takefuji Y, Lee K. An artificial hysteresis binary neuron: a model suppressing the oscillatory behaviors of neural dynamics. Biol Cybernet. (1991) 64(5):353–6.
29. Bear MF, Connors BW, Paradiso MA. Neuroscience: Exploring the Brain. 3rd ed. Philadelphia (PA): Lippincott Williams & Wilkins (2007).
30. Baptista M, Kakmeni FM, Grebogi C. Combined effect of chemical and electrical synapses in Hindmarsh-Rose neural networks on synchronization and the rate of information. Phys Rev E. (2010) 82(3):036203.
31. Liu S-C, Delbruck T, Indiveri G, Whatley A, Douglas R. Event-Based Neuromorphic Systems. John Wiley & Sons (2014).
33. Sutor B, Luhmann HJ. Development of excitatory and inhibitory. Perspect Dev Neurobiol. (1995) 2(43):409–19.
34. Stewart MG, Popov VI, Kraev IV, Medvedev N, Davies HA. Structure and complexity of the synapse and dendritic spine. In: Pickel VM, Segal M editors. The Synapse. Elsevier (2014). p. 1–20.
35. Heinbockel T. Introductory chapter: mechanisms and function of synaptic plasticity. Synaptic Plasticity. (2017). 1 p.
36. Hennig MH. Theoretical models of synaptic short term plasticity. Front Comput Neurosci. (2013) 7:45.
39. Shouval HZ, Wang SS-H, Wittenberg GM. Spike timing dependent plasticity: a consequence of more fundamental learning rules. Front Comput Neurosci. (2010) 4:19.
40. Cavallari S, Panzeri S, Mazzoni A. Comparison of the dynamics of neural interactions between current-based and conductance-based integrate-and-fire recurrent networks. Front Neur Circ. (2014) 8:12.
41. Bartolozzi C, Indiveri G. Synaptic dynamics in analog VLSI. Neur Comput. (2007) 19(10):2581–603.
42. Fusi S, Annunziato M, Badoni D, Salamon A, Amit DJ. Spike-driven synaptic plasticity: theory, simulation, VLSI implementation. Neur Comput. (2000) 12(10):2227–58.
43. Mayes DJ, Hamilton A, Louvet JE. A VLSI current mode synapse chip. International Workshop on Artificial Neural Networks. Springer (1995). p. 815–21.
44. Boahen KA. The retinomorphic approach: pixel-parallel adaptive amplification, filtering, and quantization. Anal Integr Circ Sig Proc. (1997) 13(1–2):53–68.
46. Gilbert B, Translinear B. Circuits: an historical overview. Anal IntegrCirc Sig Proc. (1996) 9(2):95–118.
47. Serrano-Gotarredona T, Linares-Barranco B, Andreou AG. A general translinear principle for subthreshold MOS transistors. IEEE Trans Circ Syst I Fund Theor Appl. (1999) 46(5):607–16.
48. Mitra S, Indiveri G, Etienne-Cummings R. Synthesis of log-domain integrators for silicon synapses with global parametric control. Proceedings of 2010 IEEE International Symposium on Circuits and Systems. IEEE (2010). p. 97–100.
49. Papadimitriou KI, Liu S-C, Indiveri G, Drakakis EM. Neuromorphic log-domain silicon synapse circuits obey bernoulli dynamics: a unifying tutorial analysis. Front Neurosci. (2015) 8:428.
50. Bartolozzi C, Indiveri G. Silicon synaptic homeostasis. Brain Inspir Cogn Syst. (2006) 2006:1–4.
52. Hasler PE, Diorio C, Minch BA, Mead C. Single transistor learning synapses. In: Tesauro G, Touretzky DS, Leen TK editors. Proceedings of the 1994 Conference; 1994 Nov 28-Dec 1; Denver, CO. Cambridge (MA): MIT Press (1995). p. 817–24.
53. Noack M, Krause M, Mayr C, Partzsch J, Schüffny R. VLSI implementation of a conductance-based multi-synapse using switched-capacitor circuits. 2014 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE (2014). p. 850–3.
54. Noack M, Partzsch J, Mayr CG, Hänzsche S, Scholze S, Höppner S, et al. Switched-capacitor realization of presynaptic short-term-plasticity and stop-learning synapses in 28 nm CMOS. Front Neurosci. (2015) 9:10.
55. Wang T, Yin L, Zou X, Shu Y, Rasch MJ, Wu S. A phenomenological synapse model for asynchronous neurotransmitter release. Front Comput Neurosci. (2016) 9:153.
56. Neff RA, Conroy WG, Schoellerman JD, Berg DK. Synchronous and asynchronous transmitter release at nicotinic synapses are differentially regulated by postsynaptic PSD-95 proteins. J Neurosci. (2009) 29(50):15770–9.
57. Ansari M, Fayyazi A, BanaGozar A, Maleki MA, Kamal M, Afzali-Kusha A et al. PHAX: physical characteristics aware ex-situ training framework for inverter-based memristive neuromorphic circuits. IEEE Trans Comput Aided Des Integr Circuits Syst. (2017) 37(8):1602–13.
59. Boybat I, Le Gallo M, Nandakumar S, Moraitis T, Parnell T, Tuma T, et al. Neuromorphic computing with multi-memristive synapses. Nat Commun. (2018) 9(1):1–12.
60. Xu L, Li C, Chen L. Analog memristor based neuromorphic crossbar circuit for image recognition. 2015 Sixth International Conference on Intelligent Control and Information Processing (ICICIP). IEEE (2015). p. 155–60.
61. Strukov DB, Stewart DR, Borghetti J, Li X, Pickett M, Ribeiro GM, et al. Hybrid CMOS/Memristor circuits. Proceedings of 2010 IEEE International Symposium on Circuits and Systems. IEEE (2010). p. 1967–70.
62. Ielmini D, Ambrogio S, Milo V, Balatti S, Wang Z-Q. Neuromorphic computing with hybrid Memristive/CMOS synapses for real-time learning. 2016 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE (2016). p. 1386–9.
63. Fayyazi A, Ansari M, Kamal M, Afzali-Kusha A, Pedram M. An ultra low-power memristive neuromorphic circuit for internet of things smart sensors. IEEE Int Things J. (2018) 5(2):1011–22.
64. Bullinaria JA. Recurrent neural networks. Neural Computation: Lecture 12.1. University of Birmingham (2013).
65. Salehinejad H, Sankar S, Barfett J, Colak E, Valaee S. Recent Advances in Recurrent Neural Networks. arXiv:1801.01078 (2017).
66. Cauwenberghs G. An analog VLSI recurrent neural network learning a continuous-time trajectory. IEEE Trans Neur Netw. (1996) 7(2):346–61.
67. Yin C, Zhu Y, Fei J, He X. A deep learning approach for intrusion detection using recurrent neural networks. IEEE Access. (2017) 5:21954–61.
68. Krenker A, Bìster J, Kos A. Introduction to the artificial neural networks. Artificial Neural Networks: Methodological Advances and Biomedical Applications. InTech (2011). p. 1–18.
69. Yu S, Gao B, Fang Z, Yu H, Kang J, Wong H-SP. Stochastic learning in oxide binary synaptic device for neuromorphic computing. Front Neurosci. (2013) 7:186.
70. Merolla PA, Arthur JV, Shi BE, Boahen KA. Expandable networks for neuromorphic chips. IEEE Trans Circ Syst I Reg Papers. (2007) 54(2):301–11.
71. Shao YS, Clemons J, Venkatesan R, Zimmer B, Fojtik M, Jiang N, et al. Simba: scaling deep-learning inference with multi-chip-module-based architecture. Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture. IEEE (2019). p. 14–27.
73. Almomani A, Alauthman M, Alweshah M, Dorgham O, Albalas F. A comparative study on spiking neural network encoding schema: implemented with cloud computing. Clust Comp. (2019) 22(2):419–33.
74. Vreeken J. Spiking neural networks, an introduction. UU-CS Report 2003–2008. Utrecht: Utrecht University, Department of Information and Computing Sciences (2003).
75. Ghosh-Dastidar S, Adeli H. A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection. Neur Netw. (2009) 22(10):1419–31.
76. Nitta T. Solving the XOR problem and the detection of symmetry using a single complex-valued neuron. Neur Netw. (2003) 16(8):1101–5.
77. Fisher RA. The use of multiple measurements in taxonomic problems. Ann Eugen. (1936) 7(2):179–88.
78. Adeli H, Ghosh-Dastidar S, Dadmehr N. A wavelet-chaos methodology for analysis of EEGs and EEG subbands to detect seizure and epilepsy. IEEE Trans Biomed Eng. (2007) 54(2):205–11.
79. Mallick PK, Ryu SH, Satapathy SK, Mishra S, Nguyen GN, Tiwari P. Brain MRI image classification for cancer detection using deep wavelet autoencoder-based deep neural network. IEEE Access. (2019) 7:46278–87.
80. Zhang L, Luo T, Zhang F, Wu Y. A recommendation model based on deep neural network. IEEE Access. (2018) 6:9454–63.
81. Ghahramani Z. Unsupervised learning. Summer School on Machine Learning. Berlin Heidelberg: Springer (2003). p. 72–112.
82. Kreiser R, Moraitis T, Sandamirskaya Y, Indiveri G. On-chip unsupervised learning in winner-take-all networks of spiking neurons. 2017 IEEE Biomedical Circuits and Systems Conference (BioCAS). IEEE (2017). p. 1–4.
83. Stuart R, Peter N. Artificial Intelligence: A Modern Approach. 2nd ed. Upper Saddle River (NJ): Prentice Hall (2003).
84. Liu T, Jiang L, Jin Y, Quan G, Wen W. Pt-spike: a precise-time-dependent single spike neuromorphic architecture with efficient supervised learning. 2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC). IEEE (2018). p. 568–73.
85. Panda S, Panda G. Fast and improved backpropagation learning of multi-layer artificial neural network using adaptive activation function. Exp Syst. (2020) 37(5):e12555.
86. Panda S, Panda G. Performance evaluation of a new BP algorithm for a modified artificial neural network. Neur Proc Lett. (2020) 51(2):1869–89.
87. Kulkarni SR, Rajendran B. Spiking neural networks for handwritten digit recognition—supervised learning and network optimization. Neur Netw. (2018) 103:118–27.
88. Panda S, Panda G. On the development and performance evaluation of improved radial basis function neural networks. IEEE Trans Syst Man Cybernet Syst. (2021) 52(6):3873–84.
89. Shakeel PM, Tobely TEE, Al-Feel H, Manogaran G, Baskar S. Neural network based brain tumor detection using wireless infrared imaging sensor. IEEE Access. (2019) 7:5577–88.
90. Sutton RS, Barto AG. Reinforcement Learning: An Introduction. 2nd ed. Cambridge, MA: MIT Press (2018).
91. Ahmed MR. Optimization of Memory Requirement Using Reinforcement Learning Based on Cognitive Science in Neuromorphic VLSI Chips. (2017).
92. Hehn M, D’Andrea R. A flying inverted pendulum. 2011 IEEE International Conference on Robotics and Automation. IEEE (2011). p. 763–70.
93. Figueroa R, Faust A, Cruz P, Tapia L, Fierro R. Reinforcement learning for balancing a flying inverted pendulum. Proceeding of the 11th World Congress on Intelligent Control and Automation. IEEE (2014). p. 1787–93.
94. Wu N, Vincent AF, Strukov D. Preliminary results towards reinforcement learning with mixed-signal memristive neuromorphic circuits. 2019 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE (2019). p. 1–5.
95. Vandesompele A, Walter F, Röhrbein F. Neuro-evolution of spiking neural networks on spinnaker neuromorphic hardware. 2016 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE (2016). p. 1–6.
96. Seo J-S, Seok M. Digital CMOS neuromorphic processor design featuring unsupervised online learning. 2015 IFIP/IEEE International Conference on Very Large Scale Integration (VLSI-SoC). IEEE (2015). p. 49–51.
97. Zhang J, Wang Y, Zhang X, Huang R. Compact digital-controlled neuromorphic circuit with low power consumption. 2017 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE (2017). p. 1–4.
98. Kim J, Koo J, Kim T, Kim J-J. Efficient synapse memory structure for reconfigurable digital neuromorphic hardware. Front Neurosci. (2018) 12:829.
99. Ahn B. Neuron-like digital hardware architecture for large-scale neuromorphic computing. 2015 International Joint Conference on Neural Networks (IJCNN). IEEE (2015). p. 1–8.
100. Yu T, Cauwenberghs G. Analog VLSI neuromorphic network with programmable membrane channel kinetics. 2009 IEEE International Symposium on Circuits and Systems. IEEE (2009). p. 349–52.
101. Mill R, Sheik S, Indiveri G, Denham SL. A model of stimulus-specific adaptation in neuromorphic analog VLSI. IEEE Trans Biomed Circ Syst. (2011) 5(5):413–9.
102. Koickal TJ, Hamilton A, Tan SL, Covington JA, Gardner JW, Pearce TC. Analog VLSI circuit implementation of an adaptive neuromorphic olfaction chip. IEEE Trans Circ Systems I Reg Papers. (2007) 54(1):60–73.
103. Mikaitis M, Lester DR, Shang D, Furber S, Liu G, Garside J, et al. Approximate fixed-point elementary function accelerator for the spinnaker-2 neuromorphic chip. 2018 IEEE 25th Symposium on Computer Arithmetic (ARITH). IEEE (2018). p. 37–44.
104. Furber SB, Galluppi F, Temple S, Plana LA. The spinnaker project. Proc IEEE. (2014) 102(5):652–65.
105. Furber SB, Lester DR, Plana LA, Garside JD, Painkras E, Temple S, et al. Overview of the spinnaker system architecture. IEEE Trans Comput. (2012) 62(12):2454–67.
106. Navaridas J, Luján M, Plana LA, Temple S, Furber SB. Spinnaker: enhanced multicast routing. Parallel Comput. (2015) 45:49–66.
107. van Albada SJ, Rowley AG, Senk J, Hopkins M, Schmidt M, Stokes AB, et al. Performance comparison of the digital neuromorphic hardware spinnaker and the neural network simulation software nest for a full-scale cortical microcircuit model. Front Neurosci. (2018) 12:291.
108. Tapiador-Morales R, Dominguez-Morales JP, Gutierrez-Galan D, Rios-Navarro A, Jimenez-Fernandez A, Linares-Barranco A. Live demonstration: neuromorphic row-by-row multi-convolution FPGA processor-spinnaker architecture for dynamic-vision feature extraction. 2019 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE (2019). p. 1–1.
109. Gutierrez-Galan D, Dominguez-Morales JP, Perez-Pena F, Jimenez-Fernandez A, Linares-Barranco A. Live demonstration: neuromorphic robotics, from audio to locomotion through spiking CPG on spinnaker. 2019 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE (2019). p. 1.
110. Mayr C, Hoeppner S, Furber S. Spinnaker 2: a 10 million core processor system for brain simulation and machine learning. arXiv preprint arXiv:1911.02385 (2019).
111. Liu C, Bellec G, Vogginger B, Kappel D, Partzsch J, Neumärker F, et al. Memory-efficient deep learning on a spinnaker 2 prototype. Front Neurosci. (2018) 12:840.
112. Lin C-K, Wild A, Chinya GN, Cao Y, Davies M, Lavery DM, et al. Programming spiking neural networks on Intel’s Loihi. Computer. (2018) 51(3):52–61.
113. Lines A, Joshi P, Liu R, McCoy S, Tse J, Weng Y-H, et al. Loihi asynchronous neuromorphic research chip. 2018 24th IEEE International Symposium on Asynchronous Circuits and Systems (ASYNC). IEEE (2018). p. 32–3.
114. Davies M, Srinivasa N, Lin T-H, Chinya G, Cao Y, Choday SH, et al. Loihi: a neuromorphic manycore processor with on-chip learning. IEEE Micro. (2018) 38(1):82–99.
115. Renner A, Evanusa M, Sandamirskaya Y. Event-based attention and tracking on neuromorphic hardware. arXiv preprint arXiv:1907.04060 (2019).
116. Frady EP, Orchard G, Florey D, Imam N, Liu R, Mishra J, et al. Neuromorphic nearest-neighbor search using Intel’s Pohoiki springs. arXiv preprint arXiv:2004.12691 (2020).
117. Imam N, Cleland TA. Rapid online learning and robust recall in a neuromorphic olfactory circuit. Nat Mach Intell. (2020) 2(3):181–91.
118. Merolla PA, Arthur JV, Alvarez-Icaza R, Cassidy AS, Sawada J, Akopyan F, et al. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science. (2014) 345(6197):668–73.
119. Moran S, Gaonkar B, Whitehead W, Wolk A, Macyszyn L, Iyer SS. Deep learning for medical image segmentation–using the IBM truenorth neurosynaptic system. Medical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications. International Society for Optics and Photonics (2018). 1057915 p.
120. Löhr MP, Jarvers C, Neumann H. Complex neuron dynamics on the IBM truenorth neurosynaptic system. 2020 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS). IEEE (2020). p. 113–7.
121. Salles A, Bjaalie JG, Evers K, Farisco M, Fothergill BT, Guerrero M, et al. The human brain project: responsible brain research for the benefit of society. Neuron. (2019) 101(3):380–4.
122. Markram H, Meier K, Lippert T, Grillner S, Frackowiak R, Dehaene S, et al. Introducing the human brain project. Proc Comput Sci. (2011) 7:39–42.
123. Dante V, Del Giudice P, Mattia M. Implementation of neuromorphic systems: from discrete components to analog VLSI chips (testing and communication issues). Annali-Istituto Superiore Di Sanita. (2001) 37(2):231–40.
124. Thanasoulis V, Partzsch J, Hartmann S, Mayr C, Schüffny R. Dedicated FPGA communication architecture and design for a large-scale neuromorphic system. 2012 19th IEEE International Conference on Electronics, Circuits, and Systems (ICECS 2012). IEEE (2012). p. 877–80.
125. Thanasoulis V, Partzsch J, Vogginger B, Schüffny R. Configurable pulse routing architecture for accelerated multi-node neuromorphic systems. 2014 21st IEEE International Conference on Electronics, Circuits and Systems (ICECS). IEEE (2014). p. 738–41.
126. Tsurumi K, Suzuki K, Takeuchi K. A 6.8 tops/w energy efficiency, 1.5 μw power consumption, pulse width modulation neuromorphic circuits for near-data computing with SSD. 2018 IEEE Asian Solid-State Circuits Conference (A-SSCC). IEEE (2018). p. 129–32.
127. Yang Z, Huo J, Monteiro H, Murray A. Sensor-driven neuromorphic walking leg control. 2009 IEEE International Symposium on Circuits and Systems. IEEE (2009). p. 2137–40.
128. Mitchell JP, Bruer G, Dean ME, Plank JS, Rose GS, Schuman CD. Neon: neuromorphic control for autonomous robotic navigation. 2017 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS). IEEE (2017). p. 136–42.
129. Subramaniam A. A neuromorphic approach to image processing and machine vision. 2017 Fourth International Conference on Image Information Processing (ICIIP). IEEE (2017). p. 1–6.
130. Scribner D, Petty T, Mui P. Neuromorphic readout integrated circuits and related spike-based image processing. 2017 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE (2017). p. 1–4.
131. Abd H, Zaman Q, Alraho S, König A. Neuromorphic spiking sensory system with self-x capabilities. IEEE Access (2024) 2:97017–38.
132. Gupta M, Kumar R, Abraham A. Adversarial network-based classification for Alzheimer’s disease using multimodal brain images: a critical analysis. IEEE Access. (2024) 12:48366–78.
133. Abreu Araujo F, Riou M, Torrejon J, Tsunegi S, Querlioz D, Yakushiji K, et al. Role of non-linear data processing on speech recognition task in the framework of reservoir computing. Sci Rep. (2020) 10(1):328.
134. Wu J, Yılmaz E, Zhang M, Li H, Tan KC. Deep spiking neural networks for large vocabulary automatic speech recognition. Front Neurosci. (2020) 14:199.
135. Wang X, Yang F, Liu Q, Zhang Z, Wen Z, Chen J, et al. Neuromorphic circuits based on memristors: endowing robots with a human-like brain. J Semicond. (2024) 45(6):061301.
136. Gupta J, Koppad D. Artificial neural network hardware implementation: recent trends and applications. International Conference On Computational Vision and Bio Inspired Computing. Cham: Springer International Publishing (2019).
137. Raymond JL, Lisberger SG, Mauk MD. The cerebellum: a neuronal learning machine? Science. (1996) 272(5265):1126–31.
138. Buonomano DV, Maass W. State-dependent computations: spatiotemporal processing in cortical networks. Nat Rev Neurosci. (2009) 10(2):113–25.
139. Goldberg Y. Neural Network Methods in Natural Language Processing. Morgan & Claypool Publishers (2017).
140. Diehl PU, Zarrella G, Cassidy A, Pedroni BU, Neftci E. Conversion of artificial recurrent neural networks to spiking neural networks for low-power neuromorphic hardware. 2016 IEEE International Conference on Rebooting Computing (ICRC). IEEE (2016). p. 1–8.
141. Rajasekharan D, Trivedi AR, Chauhan YS. Neuromorphic circuits on FDSOI technology for computer vision applications. 2019 32nd International Conference on VLSI Design and 2019 18th International Conference on Embedded Systems (VLSID). IEEE (2019). p. 504–5.
142. Orchard G, Jayawant A, Cohen GK, Thakor N. Converting static image datasets to spiking neuromorphic datasets using saccades. Front Neurosci. (2015) 9:437.
143. Han WJ, Han IS. Bio-inspired computing of vision—fuzzy and neuromorphic processing. 2011 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2011). IEEE (2011). p. 747–50.
144. Han WJ, Han I. Pedestrian detection-a neuromorphic approach. 2011 IEEE 54th International Midwest Symposium on Circuits and Systems (MWSCAS). IEEE (2011). p. 1–4.
145. Ranjan R, Ponce PM, Hellweg WL, Kyrmanidis A, Saleh LA, Schroeder D, et al. Integrated circuit with memristor emulator array and neuron circuits for biologically inspired neuromorphic pattern recognition. J Circ Syst Comput. (2017) 26(11):1750183.
146. Jiang Y, Huang P, Zhu D, Zhou Z, Han R, Liu L, et al. Design and hardware implementation of neuromorphic systems with RRAM synapses and threshold-controlled neurons for pattern recognition. IEEE Trans Circ Syst I Reg Papers. (2018) 65(9):2726–38.
© The Author(s). 2026 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.