If you're seeing this message, it means we're having trouble loading external resources on our website. Show
If you're behind a web filter, please make sure that the domains *.kastatic.org and *.kasandbox.org are unblocked. Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, London, UK Search for articles by this authorBen Rollo Affiliations Department of Neuroscience, Central Clinical School, Monash University, Melbourne, Australia Search for articles by this authorAdeel Razi Affiliations Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, London, UKTurner Institute for Brain and Mental Health, Monash University, Clayton, VIC, AustraliaMonash Biomedical Imaging, Monash University, Clayton, VIC, AustraliaCIFAR Azrieli Global Scholars Program, CIFAR, Toronto, Canada Search for articles by this authorKarl J. Friston Affiliations Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, London, UK Search for articles by this authorAuthor Footnotes 11 Lead contact Open AccessPublished:October 12, 2022DOI:https://doi.org/10.1016/j.neuron.2022.09.001 In vitro neurons learn and exhibit sentience when embodied in a simulated game-world Previous ArticleMyeloid cell interferon secretion restricts Zika flavivirus infection of developing and malignant human neural progenitor cells Next ArticleLong-range functional loops in the mouse olfactory system and their roles in computing odor identity  Improvements in performance or “learning” over time following closed-loop feedback Learning observed from both human and primary mouse cortical neurons Systems with stimulus but no feedback show no learning Dynamic changes observed in neural electrophysiological activity during embodiment SummaryIntegrating neurons into digital systems may enable performance infeasible with silicon alone. Here, we develop DishBrain, a system that harnesses the inherent adaptive computation of neurons in a structured environment. In vitro neural networks from human or rodent origins are integrated with in silico computing via a high-density multielectrode array. Through electrophysiological stimulation and recording, cultures are embedded in a simulated game-world, mimicking the arcade game “Pong.” Applying implications from the theory of active inference via the free energy principle, we find apparent learning within five minutes of real-time gameplay not observed in control conditions. Further experiments demonstrate the importance of closed-loop structured feedback in eliciting learning over time. Cultures display the ability to self-organize activity in a goal-directed manner in response to sparse sensory information about the consequences of their actions, which we term synthetic biological intelligence. Future applications may provide further insights into the cellular correlates of intelligence. Graphical abstractGraphical Abstract
Keywords
IntroductionHarnessing the computational power of living neurons to create synthetic biological intelligence (SBI), previously confined to the realm of science fiction, may now be within reach of human innovation. The superiority of biological computation has been widely theorized with attempts to develop biomimetic hardware supporting neuromorphic computing (
Third-order nanocircuit elements for neuromorphic engineering. Nature. 2020; 585: 518-523https://doi.org/10.1038/s41586-020-2735-5
Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting. The MIT Press, 2006https://doi.org/10.7551/mitpress/2526.001.0001
Third-order nanocircuit elements for neuromorphic engineering. Nature. 2020; 585: 518-523https://doi.org/10.1038/s41586-020-2735-5
Neuronal Computation Underlying Inferential Reasoning in Humans and Mice. Cell. 2020; 183: 228-243.e21https://doi.org/10.1016/j.cell.2020.08.035
Sentience and the Origins of Consciousness: From Cartesian Duality to Markovian Monism. Entropy. 2020; 22: 516https://doi.org/10.3390/e22050516
Neurons Embodied in a Virtual World: Evidence for Organoid Ethics?. AJOB Neurosci. 2022; 13: 114-117https://doi.org/10.1080/21507740.2022.2048731
From in vivo to in vitro: The medical device testing paradigm shift. ALTEX. 2017; 34: 479-500https://doi.org/10.14573/altex.1608081
The population doctrine in cognitive neuroscience. Neuron. 2021; 109: 3055-3068https://doi.org/10.1016/j.neuron.2021.07.011
Planning in the brain. Neuron. 2022; 110: 914-934https://doi.org/10.1016/j.neuron.2021.12.018
Organoids of intelligence. Nat. Phys. 2018; 14: 634https://doi.org/10.1038/s41567-018-0200-2
Figure 1DishBrain system and experimental protocol schematic Neuronal cultures derived from hiPSC via DSI protocol, NGN2 lentivirus-directed differentiation, or primary cortical cells from E15.5 mouse embryos were plated onto HD-MEA chips and embedded in a stimulated game-world of “Pong” via the DishBrain system. Different DishBrain environments were created by altering the pattern of sensory information (yellow bolts), feedback (colored bolts), or no stimulus (red crosses) to demonstrate (1 and 2) low-latency, closed-loop feedback system (stimulation (STIM) and silent (SIL) treatment); (3) no-feedback (NF) system to demonstrate an open-loop feedback configuration; and (4) rest (RST) configuration to demonstrate a system in which sensory information is absent. Interactive visualizer of activity and gameplay: https://bit.ly/3DSi4Eg.
This system, termed DishBrain, can leverage the inherent property of neurons to share a “language” of electrical activity to link silicon and BNN systems through electrophysiological stimulation and recording. Given the compatibility of hardware and cells (wetware), it is necessary to investigate what processes would result in intelligent (goal-directed) behavior when BNNs are embodied through a closed-loop system. Two interrelated processes are required for sentient behavior in an intelligent system. Firstly, the system must learn how external states influence internal states via perception and how internal states influence external states via action. Secondly, the system must infer from its sensory states when it should adopt a particular activity and how its actions will influence the environment. To address the first imperative, custom software drivers were developed to create low-latency closed-loop feedback systems that simulated exchange with an environment for BNNs through electrical stimulation. Closed-loop systems afford an in vitro culture “embodiment” by providing feedback on the causal effect of the behavior from the cell culture. Embodiment requires a separation of internal versus external states where feedback of the effect of an action on a given environment is available. Previous works, both in vitro and in silico, have shown that electrophysiological closed-loop feedback systems engender significant network plasticity (
Spatio-temporal electrical stimuli shape behavior of an embodied cortical network in a goal-directed learning task. J. Neural. Eng. 2008; 5: 310-323https://doi.org/10.1088/1741-2560/5/3/004
Shaping Embodied Neural Networks for Adaptive Goal-directed Behavior. PLoS Comput. Biol. 2008; 4: e1000042https://doi.org/10.1371/journal.pcbi.1000042
Visuomotor Coupling Shapes the Functional Development of Mouse Visual Cortex. Cell. 2017; 169: 1291-1302.e14https://doi.org/10.1016/j.cell.2017.05.023
To address the second requirement, a theoretical framework for how intelligent behavior may arise was tested by the DishBrain system. One proposition for how intelligent behavior may arise in an intelligent system embodied in an environment is the theory of active inference via the free energy principle (FEP) (
Perception and self-organized instability. Front. Comput. Neurosci. 2012; 6: 44https://doi.org/10.3389/fncom.2012.00044
The free-energy principle: a unified brain theory?. Nat. Rev. Neurosci. 2010; 11: 127-138https://doi.org/10.1038/nrn2787
On Markov blankets and hierarchical self-organization. J. Theor. Biol. 2020; 486: 110089https://doi.org/10.1016/j.jtbi.2019.110089
Generalised free energy and active inference. Biol. Cybern. 2019; 113: 495-513https://doi.org/10.1007/s00422-019-00805-w
The Discrete and Continuous Brain: From Decisions to Movement—And Back Again. Neural Comput. 2018; 30: 2319-2347https://doi.org/10.1162/neco_a_01102
Generalised free energy and active inference. Biol. Cybern. 2019; 113: 495-513https://doi.org/10.1007/s00422-019-00805-w
Previous work supports that in vitro neuronal networks can perform blind-source separation in an open-loop environment via state-dependent Hebbian plasticity consistent with the FEP (
Cultured Cortical Neurons Can Perform Blind Source Separation According to the Free-Energy Principle. PLoS Comput. Biol. 2015; 11: e1004643https://doi.org/10.1371/journal.pcbi.1004643
In vitro neural networks minimise variational free energy. Sci. Rep. 2018; 8: 16926https://doi.org/10.1038/s41598-018-35221-w
ResultsGrowth of neuronal “wetware” for computationCortical cells from the dissected cortices of rodent embryos can be grown on MEAs in nutrient-rich media and maintained for months (
Neuronal medium that supports basic synaptic functions and activity of human neurons in vitro. Proc. Natl. Acad. Sci. USA. 2015; 112: E2725-E2734https://doi.org/10.1073/pnas.1504393112
The Use of ex Vivo Rodent Platforms in Neuroscience Translational Research With Attention to the 3Rs Philosophy. Front. Vet. Sci. 2018; 5: 164https://doi.org/10.3389/fvets.2018.00164
Spontaneous periodic synchronized bursting during formation of mature patterns of connections in cortical cultures. Neurosci. Lett. 1996; 206: 109-112https://doi.org/10.1016/S0304-3940(96)12448-4
An extremely rich repertoire of bursting patterns during the development of cortical cultures. BMC Neurosci. 2006; 7: 11https://doi.org/10.1186/1471-2202-7-11
Neurons derived from human embryonic stem cells extend long-distance axonal projections through growth along host white matter tracts after intra-cerebral transplantation. Front. Cell. Neurosci. 2012; 6: 11https://doi.org/10.3389/fncel.2012.00011
Signals involved in neural differentiation of human embryonic stem cells. Neurosignals. 2009; 17: 234-241https://doi.org/10.1159/000231890
Directed differentiation of human pluripotent stem cells to cerebral cortex neurons and neural networks. Nat. Protoc. 2012; 7: 1836-1846https://doi.org/10.1038/nprot.2012.116
Neurons derived from human embryonic stem cells extend long-distance axonal projections through growth along host white matter tracts after intra-cerebral transplantation. Front. Cell. Neurosci. 2012; 6: 11https://doi.org/10.3389/fncel.2012.00011
Neural Crest Cells from Dual SMAD Inhibition: Neural Crest Cells from Dual SMAD Inhibition. in: Bhatia M. Elefanty A.G. Fisher S.J. Patient R. Schlaeger T. Snyder E.Y. Current Protocols in Stem Cell Biology. John Wiley & Sons, Inc., 2015https://doi.org/10.1002/9780470151808.sc01h09s33
Rapid generation of functional and homogeneous excitatory human forebrain neurons using Neurogenin-2 (Ngn2). Protoc. Exch. 2018; https://doi.org/10.1038/protex.2018.082
Rapid Single-Step Induction of Functional Neurons from Human Pluripotent Stem Cells. Neuron. 2013; 78: 785-798https://doi.org/10.1016/j.neuron.2013.05.029
Inducing functional radial glia-like progenitors from cortical astrocyte cultures using micropatterned PMMA. Biomaterials. 2012; 33: 1759-1770https://doi.org/10.1016/j.biomaterials.2011.10.086
Figure 2Cortical cells form dense interconnected networks (A and B) Cortical cells from E15 mouse brains and differentiated from hiPSCs, respectively. DAPI in blue stains all cells, NeuN in green shows neurons, beta III tubulin (BIII) marks axons, while MAP2 marks dendrites. Scale bar = 50μm. (C) GFAP shows supporting astrocytes, critical for long-term functioning; TBR1 marks cortex-specific cells. No Ki67, a marker of dividing cells, was observed with these cultures. Scale bar = 50μm. (D) Gene expression studies over 28 days demonstrated increased expression of the glutamatergic neural marker, vesicular glutamate transporter 1 (vGLUT1). (E–G) Neurons differentiated from hiPSCs using the DSI protocol, maintained on MEA for >3 months. White arrows show regions of shrinkage within the cultures, red arrows show bundles of axons, and blue arrows show single neurite extensions. Note the dense coverage over the HD-MEA and overlapping connections extended from neuronal soma present in all cultures across multiple electrodes. Scale bars: E = 200μm, F = 100μm, G = 50μm (H) Has false coloring to highlight the HD-MEA electrodes beneath the cells. Scale bar = 20μm.
Neural cells show well-characterized spontaneous action potentials that develop over timeIn vitro development of electrophysiological activity in neural systems at high spatial and temporal resolution was mapped. Robust activity in primary cortical cells from E15.5 rodents was found at days in vitro (DIV) 14 (A and 3E ) where bursts of synchronized activity were regularly observed, as previously demonstrated (
Spontaneous periodic synchronized bursting during formation of mature patterns of connections in cortical cultures. Neurosci. Lett. 1996; 206: 109-112https://doi.org/10.1016/S0304-3940(96)12448-4
An extremely rich repertoire of bursting patterns during the development of cortical cultures. BMC Neurosci. 2006; 7: 11https://doi.org/10.1186/1471-2202-7-11
Directed differentiation of human pluripotent stem cells to cerebral cortex neurons and neural networks. Nat. Protoc. 2012; 7: 1836-1846https://doi.org/10.1038/nprot.2012.116
Figure 3Cortical cells display spontaneous electrophysiological activity Shaded error = 95% confidence intervals. (A) Firing rate for E15.5 primary rodent cortical cells, hiPSC cells differentiated to cortical neurons via DSI, and hiPSC cells differentiated via NGN2 direct differentiation. Note different time points for each cell type. Scale bar displays firing frequency (Hz) from 0.0 to 1.0. (B) Max firing was consistently different between cortical cells from a primary source and cortical cells differentiated from hiPSCs. (C and D) Mean activity between hiPSCs differentiated using DSI and primary cortical cultures was generally similar, while hiPSCs differentiated using the NGN2 method continued to increase. This is reflected in (D), where the former two cell types displayed minimal changes in the variance in firing within a culture, while the latter increased variance over time. (E, F, and G) Showcases raster plots over 50 s, where each dot is a neuron firing an action potential colored to help distinguish channel firing and stars indicate time points with observed bursting activity. Note the differences between mid-stage cortical cells from a DIV14 primary rodent culture (E) compared with more mature DIV73 human cortical cells (F) differentiated from iPSCs using the DSI and NGN2 direct differentiated neurons (G) approach described in text, in terms of synchronized activity and stable firing patterns. While all display synchronized activity, there is a difference in the overall levels of activity represented in (B–D).
Building a modular, real-time platform to harness neuronal computationThe DishBrain system was developed to leverage neuronal computation and interact with neurons embodied in a simulated environment (; A; Video S2). The DishBrain environment is a low-latency, real-time system that interacts with the vendor MaxOne software, allowing it to be used in ways that extend its original functions (B). This system can record electrical activity in a neuronal culture and provide “sensory” (non-invasive) electrical stimulation comparably to the generation of action potentials by activity in the neuronal network (
Toward the neurocomputer: image Processing and pattern recognition with neuronal cultures. IEEE Trans. Biomed. Eng. 2005; 52: 371-383https://doi.org/10.1109/TBME.2004.842975
Figure 4Schematics and pilot testing with increasing informational density (A) Diagrammatic overview of DishBrain setup. (B) Software components and data flow in the DishBrain closed-loop system. Voltage samples flow from the MEA to the “Pong” environment, and sensory information flows back to the MEA, forming a closed loop. Full caption in Figure S2. (C) Schematic showing the different phases of stimulation to the culture. In line with this is the corresponding summed activity on the raster plot over 100 seconds. The appearance of random stimulation after a ball missing versus system-wide predictable stimulation upon a successful hit is apparent across all three representations. Corresponding images on the right show the position of the ball on both x and y axis relative to the paddle and back wall in percentage of total distance shown on the same timescale. (D) Final electrode layout schematic for DishBrain Pong-world gameplay. (E) ∗ = p < 0.05, ∗∗∗ = p < 0.001; error bars = 95% CI. Shows average rally length over three distinct experiment rounds during design of DishBrain Pong-world where each subsequent experiment provided higher density information on ball position than the previous. MCC tested over 272 sessions, n = 50 biological replicates; HCC tested over 579 sessions, n = 18 biological replicates.
The initial proof of principle using DishBrain was to simulate the classic arcade game “Pong” by delivering inputs to a predefined sensory area of 8 electrodes (D). Electrodes were arranged in a manner that would allow a coarse, yet topographically consistent, place coding, consistent with in vivo systems (see ) (
Topographic cues of nano-scale height direct neuronal growth pattern. Biotechnol. Bioeng. 2012; 109: 1791-1797https://doi.org/10.1002/bit.24444
Topographic organization in the brain: searching for general principles. Trends Cogn. Sci. 2014; 18: 351-363https://doi.org/10.1016/j.tics.2014.03.008
The Structure of Multi-Neuron Firing Patterns in Primate Retina. J. Neurosci. 2006; 26: 8254-8266https://doi.org/10.1523/JNEUROSCI.1282-06.2006
Increasing the density of sensory information input leads to increased performanceThe DishBrain protocol was refined over three pilot studies, each increasing the density of sensory information. Pilot study 1 operated with a 4Hz stimulation that only involved place coding, where the location of the stimulation corresponded to the position of the ball on the y axis. Pilot study 2 investigated different configurations and introduced activity-based weighting to motor regions to account for cell density or activity differences. Pilot study 3 adopted the layout in D and changed to the combined rate (4–40Hz) and place-coding method of data input. This combined rate and place coding has compelling biological similarities conceptually to the rodent barrel cortex, suggesting this encoding is physiologically coherent (
An elaborate sweep-stick code in rat barrel cortex. Sci. Adv. 2020; 6: eabb7189https://doi.org/10.1126/sciadv.abb7189
Cellular and Circuit Mechanisms Maintain Low Spike Co-Variability and Enhance Population Coding in Somatosensory Cortex. Front. Comput. Neurosci. 2012; 6https://doi.org/10.3389/fncom.2012.00007
Population Coding of Stimulus Location in Rat Somatosensory Cortex. Neuron. 2001; 32: 503-514https://doi.org/10.1016/S0896-6273(01)00481-0
BNNs learn over time when embodied in a gameplay environmentTo test the predictions of the FEP (A ) using selected parameters (), cortical cells (MCCs and HCCs) were compared with media-only controls (CTL); rest sessions (RST), where active cultures controlled the paddle but received no sensory information; and in-silico (IS) controls that mimicked all aspects of the gameplay except the paddle were driven by random noise over 399 test sessions (80-CTL [n = 6 MEA], 42-RST [n = 20 cultures], 38-IS [n = 3 seeds], 101-MCCs [n = 9 cultures], 138-HCCs [n = 11 cultures]). The average rally length showed a significant interaction (B and Table S1) between group and time (first 5 and last 15 min). Only the MCC and HCC cultures showed evidence of learning with significantly increased rally lengths over time. Further, it was found that during gameplay in timepoint 1 (T1), key significant differences were observed (Table S1): the HCC group performed significantly worse than MCC, CTL, and IS groups (Table S2). This suggests that HCCs perform worse than controls when first embodied in an environment, suggesting an initial maladaptive control of the paddle or perhaps an exploratory behavior. Notably, at timepoint 2 (T2), this trend was reversed; the MCC and HCC groups significantly outperformed all control groups along with HCC showing a slight but significant outperformance over the MCC group (Tables S1 and S2). This data demonstrates a significant learning effect in both experimental groups absent in the control groups, along with evidence that the learning capabilities differ between mice and human cells in line with previous results (Video S1). Figure 5Embodied cortical neurons show significantly improved performance in “Pong” when embodied in a virtual game-world 399 test-sessions were analyzed with biological replicates: 80-CTL (n = 6), 42-RST (n = 20), 38-IS (n = 3), 101-MCCs (n = 9), 138-HCCs (n = 11). Significance bars show within-group differences denoted with ∗. Symbols show between-group differences at the given timepoint: # = versus HCC; % = versus MCC; ˆˆ = versus CTL; @ = versus IS. The number of symbols denotes the p value cutoff, where 1 = p < 0.05, 2 = p < 0.01, 3 = p < 0.001, and 4 = p < 0.0001. Boxplots show interquartile range, with bars demonstrating 1.5× interquartile range, the line marks the median, and ▲ marks the mean. (A) Schematic of how neurons may engage in the game-world under active inference denoting a gradient flow on variational free energy, expressed in terms of neural activity minimizing prediction errors. ε is prediction error, ξ represents a precision-weighted prediction error. Precision can be regarded as a Kalman gain in Kalman filtering; ‘a’ corresponds to action. (B–D) Experimental groups according to time point 1 (T1; 0–5 min) and time point 2 (T2; 6–20 min). (B) Average performance between groups over time, where only experimental (MCC: t = 6.15, p = 5.27−08 and HCC: t = 10.44, p = 3.92−19) showed significant improvement and higher average rally length against all control groups at T2. (C) Average number of aces between groups and over time, only MCC (t = 2.67, p = 0.008) and HCC (t = 5.95, p = 2.13−08) differed significantly over time. The RST group had significantly more aces compared with the CTL, IS, MCC, and HCC groups at T1 and compared with the CTL, MCC, and HCC at T2. Only MCCs and HCCs showed significant decreases in the number of aces over time, indicating learning. At T2 they also showed fewer aces compared with the IS group, but only the HCC group was significantly less than CTL. (D) Average number of long rallies (>3) performed in a session. At T1, the HCC group had significantly fewer long rallies compared with all control groups (CTL, IS, and RST). However, both the MCC (t = 5.55, p = 2.36−07) and HCC (t = 10.38, p = 5.27−19) groups showed significantly more long rallies over time. By T2, the HCC group displayed significantly more long rallies compared with the IS group. The HCC group also displayed significantly more long rallies compared with all CTL, IS, and RST control groups. (E) The average distance that the paddle moved during a session was found to have no obvious relationship with average rally length as the IS control groups showed a higher movement than the experimental groups, while CTL and RST were lower. As such, the observed learning effects are not likely due to stimulation, leading to increased activity of paddle movement. (F) Distribution of frequency of mean summed hits per minute among groups shows obvious differences; scale bar shows the probability the number of hits in the given minute under that condition.
Learning effects in BNNs are observed across additional measuresOther key gameplay characteristics, such as the number of times the paddle failed to intercept the ball without a single hit defined as “aces,” and the number of gameplays with greater than 3 consecutive hits defined as “long rallies,” were calculated. As with average rally length, significant interactions between groups and time were found for aces and long rallies (Table S1). Only the MCC and HCC groups showed significantly fewer aces in T2 compared with T1 (C and Table S2). Likewise, only the MCC and HCC groups showed significantly more long rallies in T2 compared with the first (D and Table S2). Collectively, the data shows that both experimental cultures (HCCs and MCCs) improved performance by reducing how often they missed the initial serve and achieving more consecutive hits or longer rallies. Differences between groups at T1 were found both for aces and long rallies (Table S1). The RST condition displayed significantly more aces than the CTL and MCC groups (Table S2), suggesting a degree of sporadic behavior that the cells exhibit when initially introduced to the rest period from gameplay that results in this behavior. When the number of long rallies at T1 was investigated, it was found that only HCCs had significantly fewer long rallies (Table S2). This finding complements the reduced average rally lengths discussed above. Significant differences between groups at T2 were also found for aces and long rallies (C and 5D and Table S1). Notably, the HCC group showed significantly fewer aces than CTL, RST, and IS groups (Table S1). The MCC group also showed significantly fewer aces than RST and IS groups, but not the CTL group (Table S2). In contrast, for long rallies, the MCC group showed significantly more than the CTL, RST, and IS groups (Table S2), yet the HCC group only showed significantly more long rallies compared with the IS group, but not RST or CTL (Table S2). No learning effect was found in electrically inactive non-neural cells (HEK293T cells) and media-only controls (Figures S4A–S4C). Further, a significant negative correlation between percentage of aces and percentage of long rallies of both MCCs and HCCs was found, suggesting that the performance was not arising from maladaptive behavior such as fixing the paddle to a single corner (Figure S4D). Whether stimulation alone may cause greater movement of the paddle and that this may result in the observed learning effects was also investigated. As E shows, while there were significant differences observed in paddle movement between conditions (Table S1), for the CTL and RST, this resulted in significantly lower movement relative to the other groups, with the RST being the lowest movement of all groups (Table S2). The IS control group showed significantly more paddle movement than all other groups yet displayed no meaningfully different performance metrics to the other control groups (CTL and RST) (Table S2). Additionally, Figure S4E shows no significant correlation between paddle movement and average rally length was observed, supporting that movement alone of the paddle does not explain the observed learning effects. Wholistically, F emphasizes that both MCCs and HCCs showed fewer aces and more long rallies in T2 compared with T1, reiterating the observed learning effect over time. This can also be seen in linear regressions (Figure S4F), where only the MCC and HCC groups showed a statistically significant positive relationship between average rally length and duration of gameplay. BNNs require feedback for learningTo investigate the importance of the feedback type for learning, cultures, both MCCs and HCCs, were tested under 3 conditions for 3 days, with 3 sessions per day resulting in a total of 486 sessions. Condition 1 (Stimulus; n = 27) mimicked that used above, where predictable and unpredictable stimuli were administered when the cultures behaved desirably or not, respectively. Condition 2 (Silent; n = 17) involved the stimulus feedback being replaced with a matching time period in which all stimulation was withheld, after which the game restarted with the ball beginning in a random direction. Condition 3 (No feedback; n = 15) removed the restart after a miss. When the paddle did not successfully intercept the ball, the ball would bounce and continue without interruption; the stimulus reporting ball position was still provided. The difference between these conditions is illustrated in A . Rest-period activity was also gathered and used to normalize performance per session basis to account for differences in unstimulated activity (). Figure 6The importance of feedback in learning 486 sessions were analyzed. Significance bars show within-group differences denoted with ∗. Symbols show between-group differences at the given timepoint: # = versus Stimulus; % = versus Silent. The number of symbols denotes the p value cutoff, where 1 = p < 0.05, 2 = p < 0.01, 3 = p < 0.001, and 4 = p < 0.0001. Box plots show interquartile range, with bars demonstrating 1.5× interquartile range, the line marks the median, and ▲ marks the mean. Errors bands = 1 SE. (A) Schematic showing the stimulation from the 8 sensory electrodes across 40 s of the same gameplay for each of the three conditions. The bar below color codes what phase of stimulation is being delivered, where random stimulation follows a miss and predictable stimulation follows a hit in the Stimulus condition. Note the corresponding absence of any stimulation in the Silent condition and the lack of any change in sensory stimulation in the No-feedback condition. (B) Displays the probability of a certain number of hits occurring in a group at a specific minute. (C) Using different feedback schedules, the Stimulus feedback condition showed significant learning (as in A; t = 7.48, p = 1.58−12) and outperformed Silent and No-feedback average rally length. Silent feedback also showed higher performance compared with these groups at T2. (D) Displays difference seen in (C) across day. (E) Shows similar differences versus rest performance for aces across conditions, where the Stimulus group showed significantly fewer aces across time (t = 3.21, p = 0.002). (F) Displays data from (E) across day. (G and H) Shows that the Stimulus condition showed significant increase (t = 3.21, p = 0.002) across timepoints; however, as in (H), no differences were found across time for long rallies.
Stimulus and Silent conditions showed an overall higher average rally length compared with Rest and No-feedback conditions (B). When testing for differences between groups in the percentage increase of average rally length over matched rest controls, a significant interaction was found (C and Table S1). Only the Stimulus condition showed a significant increase in average rally length over time. While no differences were found for T1, a significant main effect of group was found at T2, where the Stimulus condition had a significantly higher average rally length than the Silent and No-feedback conditions (Table S2). Interestingly, the Silent condition also significantly outperformed the No-feedback conditions, although with a smaller effect size (Table S2). Importantly, this demonstrates that information alone is insufficient; feedback is required to form a closed-loop learning system. When followed up at the level of day for T2 (D), no significant differences over time were observed, but the same between-group differences as above were observed. This trend was similar when looking at aces both summed (E) and across days of testing (F). The Stimulus group at T1 showed significantly fewer long rallies compared with the Silent and No-feedback condition, being reversed at T2 with the Stimulus group showing significantly more long rallies compared with the No-feedback condition (G). No difference was found when this was followed up across days (H). Collectively, these results suggest that adaptive behavior seen in BNNs altering electrophysiological activity can be an emergent property of engaging with—and implicitly modelling—the environment. Dynamics in electrophysiological activity display coherent connectivityElectrophysiological activity during gameplay was analyzed from cultures subjected to the stimulus condition to determine functional connectivity (
Network analysis of hippocampal neurons by microelectrode array in the presence of HIV-1 Tat and cocaine. J. Cell. Physiol. 2018; 233: 9299-9311https://doi.org/10.1002/jcp.26322
Figure 7Electrophysiological activity during Gameplay and Rest 579 sessions (358 Gameplay, 221 Rest) were analyzed with n = 43 biological replicates. Significance bars show within-group differences denoted with ∗. Symbols show between-group differences at the given timepoint: # = versus Gameplay or Stimulus; % = versus Silent. The number of symbols denotes the p-value cutoff, where 1 = p < 0.05, 2 = p <0.01, 3 = p < 0.001, and 4 = p <0.0001. Box plots show interquartile range, with bars demonstrating 1.5× interquartile range, the line marks the median, and ▲marks the mean. Error bands = 1 SE. (A–D) A significant positive correlation between mean firing and performance was found between motor region 1 and 2 with the Sensory area both during Rest (A and B) and Gameplay (C and D). (E) The average cross-sensory motor correlation was significantly less during Rest, both for motor region 1 (t = 30.40, p = 6.61−194) and motor region 2 (t = 29.76, p = 2.76−186) than during Gameplay. (F) The percentage of mutually exclusive activity events per second across motor regions was calculated and found to increase significantly during Gameplay versus Rest (t = 14.64, p = 5.68−48). (G) The correlation between the two motor regions showed substantial changes over time (blue). Linear regression conducted on the first 5 min of Gameplay (orange) showed a significant negative relationship between variables that was absent in the final 15 min (teal). (H) Activity over time showed no significant changes while engaged in Gameplay (r = −0.01, p = 0.563), supporting that any observed learning effects over time were not related to merely gross changes in activity levels across the cultures over time. (I) Functional plasticity was assessed across cultures when engaged in Gameplay versus Rest, with a significant increase in functional plasticity found during gameplay. (J) Following random stimulation feedback, there was a significant increase in the mean information entropy during Gameplay (t = 4.890, p = 2.024−6), yet the corresponding time during Rest showed no change (t = 0.016, p = 0.987). Mean information entropy was lower at both pre- (t = 9.781, p = 3.882−19) and post- (t = 5.915, p = 1.178−8) feedback during Gameplay than at Rest. (K) For normalized mean information entropy, the difference relative to feedback period was increased during Gameplay (t = 19.337, p = 3.476−48), yet still no difference was observed during Rest where no feedback was delivered (t = 1.022, p = 0.316). Normalized mean information entropy was lower at pre- (t = 10.192, p = 2.139−20), but not post- (t = 0.671, p = 0.503) feedback, during Gameplay compared with Rest. (L) Feedback-related changes in normalized mean information entropy were assessed for the investigation of different feedback mechanisms. Increases following random feedback for the Stimulus condition were replicated (t = 9.623, p = 7.887−19); it was also found that the system displayed increased activity-related scores under the Silent condition feedback (t = 21.538, p = 7.019−47). The No-feedback condition showed no change in normalized mean information entropy at matched times after Bonferroni corrections (t = 10.192, p = 0.030). Post-hoc follow-up tests found no differences between Stimulus and Silent conditions during gameplay; both were significantly lower than for the No-feedback condition. After feedback, the Stimulus and Silent conditions were significantly higher than the No-feedback condition, with the Silent condition significantly higher than the Stimulus condition.
As electrical stimulation of neural tissue has been shown to modify neuronal activity (
Spatio-temporal electrical stimuli shape behavior of an embodied cortical network in a goal-directed learning task. J. Neural. Eng. 2008; 5: 310-323https://doi.org/10.1088/1741-2560/5/3/004
Long-Term Activity-Dependent Plasticity of Action Potential Propagation Delay and Amplitude in Cortical Networks. PLoS One. 2008; 3: e2088https://doi.org/10.1371/journal.pone.0002088
Shaping Embodied Neural Networks for Adaptive Goal-directed Behavior. PLoS Comput. Biol. 2008; 4: e1000042https://doi.org/10.1371/journal.pcbi.1000042
We repeated this analysis on the follow-up study of different feedback mechanisms reported in . While it is important to note that the internal information entropy of the culture is not necessarily and directly tied to the external (i.e., sensory) information entropy of the stimulus being applied into a culture, it is interesting to see how cultures respond to different feedback protocols. As shown in L, the change during the stimulus condition between the normalized mean information entropy was replicated for the standard Stimulus condition (Table S1). Of interest is the finding that during the Silent condition, the neural cultures had a higher normalized mean information entropy than even the stimulus condition post-feedback. However, the No-feedback condition showed no change relative to the period when feedback would have been applied, with a significantly higher normalized mean information entropy score than either of the other two conditions pre-feedback, yet a significantly lower score post-feedback (Table S2). Electrophysiological activity is linked with higher average rally lengthExploratory uncorrected Pearson’s correlations were computed for key electrophysiological activity metrics and average rally length. A significant positive correlation was found between average rally length with mean (A ) and max (B) firing. Likewise, the cross-correlations with the sensory region for both motor region 1 (C) and 2 (E) were significantly positively correlated with performance, further suggesting that robust connectivity is linked with better gameplay outcomes. To further investigate whether the topographical distribution of activity correlated with performance, the absolute values of four discrete cosine transform (DCT) coefficients normalized to mean activity were used to summarize spatial modes of spontaneous activity and assess the symmetry of activity (E). DCT 0,1, which measures activity across the horizontal plane (F), and DCT 2,0, which measures activity on the horizontal edge versus the horizontal center (I), were significantly negatively correlated with average rally length. Yet, DCT 0,2, which shows difference between activity on the vertical edges and the vertical center (G), and DCT 1,0 which measures activity across the vertical plane (H), did not significantly correlate. Given configuration layout, it is coherent that gameplay performance is closely linked to deviations in symmetry of electrophysiological activity. To confirm the importance of symmetry, gameplay electrophysiological activity was analyzed for both motor regions, and the normalized deviation away from symmetry was calculated. As deviation away from symmetry resulted in a significant negative correlation with the average rally length, any asymmetry exceeding approximately 1 deviation appeared to completely prevent performance above that observed in controls (J). This suggests a limit to which cultures can self-organize spontaneous activity if cell culture quality is uneven. Finally—in line with the results above—higher activity in the sensory region (K), motor region 1 (L), and motor region 2 (M) during gameplay was also correlated with higher average rally lengths. Figure 8Relationship between electrophysiological activity and average rally length 302 gameplay sessions were analyzed after filtering outliers (Z score > ±3.29) from rallies with n = 30 biological replicates. (A) The mean spontaneous activity (Hz) over all electrodes showed a significant positive correlation with average rally length. (B–D) Similarly, the max spontaneous firing (Hz) also showed a significant positive correlation with average rally length. In line with this, the average cross correlation between the sensory region and both motor region 1 (C) and motor region 2 (D) had a significant positive correlation with average rally length. (E) The DCT scores of four different basis functions were calculated to quantify asymmetry in spontaneous activity. DCT scores were normalized to mean activity. The scale bar shows the value assigned to activity in the given area, where each DCT basis function quantifies a different type of asymmetry per pixel from −0.010 to 0.010. (F–H) Displays the significant negative correlation between DCT 0,1 and average rally length, showing that asymmetry on the horizontal axis is related to poorer performance. There was no significant relationship between DCT 0,2 (G), which measured asymmetry on the horizontal extremes compared with the center, or DCT 1,0 (H), which measured asymmetry on the vertical axis. (I–M) DCT 2,0 function displayed a significant negative correlation with average rally length, suggesting that asymmetry on the vertical edges compared with the middle was linked to poorer gameplay performance. In line with this, (J) displays the calculated deviation from symmetry in activity between motor regions during gameplay and finds a significant negative association, where greater asymmetry was linked to lower average rally lengths. Similarly, during gameplay the activity in the sensory (K), motor region 1 (L), and motor region 2 (M) all showed significant positive correlations with average rally length.
DiscussionHere, we present the DishBrain system, a system capable of embodying BNNs from various sources in a virtual environment and measuring their responses to stimuli in real time. The ability of neurons, especially in assemblies, to respond to external stimuli adaptively is well established in vivo as it forms the basis for all animal learning (
Visuomotor Coupling Shapes the Functional Development of Mouse Visual Cortex. Cell. 2017; 169: 1291-1302.e14https://doi.org/10.1016/j.cell.2017.05.023
The Tolman-Eichenbaum Machine: Unifying Space and Relational Memory through Generalization in the Hippocampal Formation. Cell. 2020; 183: 1249-1263.e23https://doi.org/10.1016/j.cell.2020.10.024
Most significantly, this work presents a substantial technical advancement in creating closed-loop environments for BNNs (
Spatio-temporal electrical stimuli shape behavior of an embodied cortical network in a goal-directed learning task. J. Neural. Eng. 2008; 5: 310-323https://doi.org/10.1088/1741-2560/5/3/004
Shaping Embodied Neural Networks for Adaptive Goal-directed Behavior. PLoS Comput. Biol. 2008; 4: e1000042https://doi.org/10.1371/journal.pcbi.1000042
Effective parameters for stimulation of dissociated cultures using multi-electrode arrays. J. Neurosci. Methods. 2004; 138: 27-37https://doi.org/10.1016/j.jneumeth.2004.03.005
It is interesting to note, however, that the internal information entropy of BNN activity does not exactly mirror the information entropy of the external stimulation: while the unpredictable stimulus increased internal entropy, so did the Silent condition feedback. However, for a BNN to alter activity in response to feedback, there must be a change to its sensory input observable by the system that can be associated with its previous activity. This is consistent with the absence of learning in the open-loop/No-feedback condition, which by its nature affords no opportunity for learning, and likewise showed higher internal information entropy than the other two feedback conditions. This supports the thesis that stimulation alone is insufficient to drive learning: there must be a motivation for learning behaviors that influence the (external) observable stimulus. When faced with unpredictable sensorium, playing “Pong” successfully acts as a free energy-minimizing solution. Even if the internal information entropy of a system is increased following feedback and has lower external information entropy (e.g., silent feedback), this may not provide the same impetus for learning. These findings accord with the proposed role of a Markov blanket, providing a statistical boundary of the system to separate it into internal and external states (
The Markov blankets of life: autonomy, active inference and the free energy principle. J. R. Soc. Interface. 2018; 15: 20170792https://doi.org/10.1098/rsif.2017.0792
On Markov blankets and hierarchical self-organization. J. Theor. Biol. 2020; 486: 110089https://doi.org/10.1016/j.jtbi.2019.110089
Mechanistically, we sought to demonstrate the utility of the DishBrain by testing base principles that underwrite active sensing via the FEP. The closest previous work examined blind source separation in neural cultures, yet did so in an open-loop context without physiologically plausible training (
Cultured Cortical Neurons Can Perform Blind Source Separation According to the Free-Energy Principle. PLoS Comput. Biol. 2015; 11: e1004643https://doi.org/10.1371/journal.pcbi.1004643
In vitro neural networks minimise variational free energy. Sci. Rep. 2018; 8: 16926https://doi.org/10.1038/s41598-018-35221-w
High frequency neurons determine effective connectivity in neuronal networks. Neuroimage. 2018; 166: 349-359https://doi.org/10.1016/j.neuroimage.2017.11.014
Different Patterns of Electrical Activity Lead to Long-term Potentiation by Activating Different Intracellular Pathways. What refers to tiny spaces between neurons?The space between the dendrites of one neuron and the axon of another neuron is called the synapse.
Where is the synaptic gap?The synaptic cleft, also known as the synaptic gap, is the space in between the axon of one neuron and the dendrites of another and is where the electrical signal is translated to a chemical signal that can be perceived by the next neuron.
What is the small gap between neurons quizlet?Synapse: The synapse is a small gap between neurons that acts as a place where communication occurs between neurons. Receptors: The site where neurotransmitters attach to the proteins on the cell surface attach.
What is the gap between neurons called quizlet?The tiny gap at this junction is called the synaptic gap or cleft. Chemical messengers that, when released by the sending neuron, travel across the synapse. (the gap between two neurons) and bind to receptor sites on receiving neurons, setting up the next link in the chain of communication within the nervous system.
|
The small space between the sending neuron and the receiving neuron is the
Related Posts
Copyright © 2024 ShotOnMac Inc.
