Human Learning Optimization

Introduction

In the 21st century, at the dawn of digital revolution we are living in the age of complexity. Where the dynamics of the complex systems are high dimensional, non-linear, partially observable and it’s hard to describe and model the aspects of reality at the appropriate level of abstraction, and it’s even harder to predict and anticipate the emergence and its outcome because of inherent uncertainty of the system. And yet, here we are, in the midst of information explosion and technological advancement. How can we manage the increasing complexity efficiently and be more powerful, thoughtful, resourceful and creative?

One approach is to understand the complexity, and understand what makes you understand the complexity would be the first logical thing to do. This means understanding our brain that generates our mind. It’s good to note that there are many things that we don’t know about the brain and mind, but thanks to the advances in technology, now the researchers are able to do experiments that they couldn’t do before. **The publiction rate of neuroscientific research paper are doubling every year. Which means that our scientific knowledge about the brain is expanding exponentially. And by reverse engineering how our brain and mind works enables us to leverage that knowledge in order to use it effectively.

The objective here is to present the core concepts and models from cognitive science and neuroscience that will makes us more adapted to survive in the age of complexity by understanding the principle and mechanism of learning. The idea of interaction and information is fundamental to learning, because we learn by interacting with the environment and encode information in memory by observing the environment. In the age of complexity, we choose how do we evolve and adapt by consciously selecting what information to encode in our brain, shaping our own development, knowledge and skills, just like games by accumulating experience points and allocating skills points where applicable. We can learn anything we want, it just requires time, energy, patience, persistence, and discipline.

You take responsibility to design knowledge and skill tree youself. It just happens that human Input/output are slow and learning process is delicate and sometimes painful. But understanding the underlying mechanism of learning and memory to self-monitor the learning progress will make the learning process less painful. Certainly, optimal path to learning exists, but I don’t think that this information here will make things easy to learn, I’m not sure that’s even possible. There is no shortcut, only the energy and willpower to make it through all the challenges will be rewarded.

Here I’ll cover the core components of biological brain and mechanism of learning and memory, focusing on the systems level description. Because, well, we already have scope that is extremely complex and the resolution of the details are technically endless as our knowlede expands. Maybe in the future, I’ll make a posts concerning each components in more detail; Until then, you can take this post as a general overview of topics.

Brain Architecture

Our brains are extremely complex in its structure and function. The architecture of our brains are hierarchical and modular network of interconnected neurons organized into cortical layers and columns. The cerebrum comprises two cerebral hemispheres, each consisting of a heavily folded outer layer called cerebral cortex, and three deep structures: the basal ganglia, the hippocampus, and the amygdala. The cerebral cortex is divided into four distinct lobes: frontal, parietal, occipital and temporal.

Each lobe of ceberal cortex has a specialized set of functions. The frontal lobe is concerned with short-term memory, planning future actions and control movements; the parietal lobe with somatic sensations, with forming a body image and relating it to space; the occipital lobe with vision and temporal lobe with hearing and through its deep structures, the basal ganglia with motor control, the hippocampus and amygdala with learning, memory and emotion.

Neural Network

There are two classes of cells that makes up the core building blocks of nervous system. The neurons, a specialized cells that sends electro-chemical signals called action potential to communicate with target cells and make a structure called synapse that regulates the synaptic transmission and connection strength between neurons to think and act. Neurons takes input signal from dendrite and produce output signal from axon to other neuron, and neurons that fire together wire together, forming a stronger connection between neurons by its association, lowering the threshold to fire again. The human brain contain a huge number of neurons, on the order of 10^11 neurons, that can be classified into at least a thousand different types. One key principle of brain is that neurons with similar properties can produce different actions because of the way they are interconnected.

The glial cells perform a number of critical functions, including structural support, metabolic support, insulation, and guidance of development. The glial cells greatly outnumber neurons. There are 2 to 10 times more glial cells than neurons.

Then, these neurons are wired together to form a neural circuit, and the cluster of neural circuits makes a massive neural network. The neural network dynamics are parallel distributed information processing that make these cluster of neural circuits to fire together by making a firing pattern to perform a specific function that related to what we do consciously or uncounsciously, like seeing, speaking, moving, and so forth. It seems that our brains are an enourmous network of interconnected nodes that constantly adapt and reconfigure.

Representation and Computation

Everything we see and hear that become things and stuffs we know and believe about the world are information. These informations are encoded and stored as memories and later retrieved by our brain when we remember the things and stuffs. The encoding process anatomically changes the neural circuitry to integrate the new information. Then these memories are represented as knowledge or skill, and finally computed by our mind to do things and stuffs that we do.

The mental representation is information such as idea, concept or symbol that represents external reality. And mental computation is mental process that makes use of such information. Generally, we call this a mental model (model of what exists and how worlds works in our mind)

Evolution and Learning Algorithm

The evolutionary algorithm is an optimization algorithm that maximize the fitness of the organism by reproduction, mutation, recombination and natural selection. Natural selection make the appropriate adaptations while leaving out the inappropriate ones, in addition to the random mutation that explore the adjacent possible, in order to maximize the survival and reproduction of the organism in whatever environment the organism is living in. Run the evolutinary algorithm over the millions of years, and the result is the complex nervous systems including the brain we have today. But the disadvantage is that the process of evoltion is very slow, and it takes milions of years to find the optimum solution to its environment.

The solution that evoltuionary algorithm devises is a learning algorithm to overcome the time it takes to evolve and adapt the organism. So, the evolutionary algorithm is an optimization algorithm, and our learning algorithm is the product of the evolutionary algorihtm, meaning that we are on the level of the learning algorithm that learns to improve on itself recursively by artificially selecting and solving ever more complex problems. And this leads to the core question: How do we optimize our learning algorithm?

The assumtpion here is that our brain and mind is a system of organs of computation equipped with learning algorithm designed by natural selection to solve problems. In which we can call, the universal learning machine. (Think of children learning to see, interact and speak, formulating complex sentences from observation and interaction without attending advanced linguistics class in wherever they happened to exist.)

Learning

Learning is a process of encoding new or modifying existing information and the outcome of learning is a memory. That is, a memory is created when something is learned, and the learning may occur either by a single exposure, or by repetition of information, experiences, or actions.

When we are learning something, we takes input from our perception constructed by sensory system as sensory memory, like visual input as 3-dimensional scene or symbol, and auditory input as soundwaves where the attention system priorize the processing of information by focusing on relevant aspect while efficiently allocate limited computational resources.

Then, the sensory memory is loaded into working memory, where informations is acquired and can be maintained and manipulated in the short-term period. At the same time, the long-term memory is activated to gradually consolidate the information over time from working memory, and then with rehearsal we can retain the encoded information for longer period of time, so that later, when we want to remember the information, we can retrive from long-term storage.

During the learning, we construct meaning out of prior knowledge about the topic, the more we know about the topic, the easier it gets to learn advanced topic. By bulding on prior understanding to learn new idea or concept facilitates the integration such that we can incrementally solve more complex problems.

The learning is an iterative process of gradually improving the sophistication of mental representation and computation of the given task by performing a problem solving at the right level of difficulty such that the performance on the given task improves with the experience.

At the core, learning is a loop of downloading data, algorithms and heuristics into the brain, then applying these heuristics based on data to make the problem tractable, then executing the algorithm to solve problems by inference, or to create new problems.

Again, the input here is information, the process is enconding and consolidating that information by interaction, which then integrate and store to the body of knowledge, and finally the output is the acquisition of new knowledge by retrieving stored knowledge or performing a skill.

Memory

Memory is the ability to encode, store, and retrieve information from past experiences in the brain. Memory is what we remember from past and imagine the future. Remebering is a reconstructive process in which fragment of memories are re-assembled from past experiences, such as remembering scenes, events, entities, objects and its interactions. Imagining is a constructive process where are assembled also from past experiences and imagined experiences to create the possible or impossible in the reality. For instance, we can invent new concepts and things, and construct a theory or build a system. It is the core capability that makes the learning and adaptation possible from previous experiences.

Memory at its simplest is a set of encoded neural connections in the brain. It is the on-the-fly reconstruction of past experiences by the synchronous firing of neurons that were involved in the original experience that makes us remember. And fundamentally there is no distinction between remembering, imagining or thingking.

We have multiple memory systems that process and store different type of information in different part of the brain. There are at least three types of memory as categorized by its capacity and duration of retention.

Information Processing Model

It is a hierarchical conceptual model of memory. - Sensory memory - Iconic memory - Echoic memory - Working memory - Long-term memory - Explicit memory - Semantic memory - Episodic memory - Implicit memory - Procedural memory

Sensory Memory

Sensory memory retain sensory information. Visual and auditory inputs are called iconic memory and echoic memory respectively. Iconic memory lasts for < 0.5 seconds and echoic memory lasts for 3-4 seconds.

Working Memory

Working memory is also referred as short-term memory, where we maintain and manipulate information. It has four components, central executive system that control Visuo-spatial sketchpad and Phonological loop, to manipulate mental imagery and sound. And Episodic Buffer which binds information from long-term memory into an unitary representation. It has limited capacity of 7+-2 information.

Long-term Memory

Long-term memory retain semantic and episodic information over long time, or even lifetime. Long-term memory has two types of memories, explicit memory and implicit memory.

Explicit memory

The explicit memory is a recollection of conciously expressible facts and events, and can be further divided into two categories, semantic memory and episodic memory.

Semantic memory

Semantic memory refers to acontextual factual knowledge about the world acquired during an experience which then becomes separated from a specific context of the learning event itself. This type of memory contributes to the formation and long-term representation of abstracted knowledge such as concepts, categories, facts, and word meanings. It also includes knowledge about ourselves.

Episodic memory

Episodic memory is the autobiographical episodes or specific events which necessarily includes information about both the content of the experience and the spatial-temporal context in which it occurred - the “what, when, where”.

Implicit memory

The implicit memory is the knowledge that we have no consious access.

Procedural memory

Procedural memory is one form of implicit memory that depends on extensive and repeated experience, for instance, learning motor skills like how to type, and cognitive skills like how to read.

Memory Operations

  • Encoding
    • Acquiring, Consolidating
  • Storing
  • Retrieving

In any type of memory, there is a general process to encode, store, and retrieve memory.

Encoding

Encoding is the first stage of processing where stream of new information creates memory traces to be stored. The sensory systems perceives the entity or object of interest to be converted into a memory that can be stored within the brain, and enable the retrival later from short-term or long-term memory. The encoding process can be broken down in two parts, adcquisition and consolidation.

Acquisition

The acquisition phase is where sensory system perceive the environment as input, then the attention system controls the flow of information processing (regulated by thalamus and frontal lobe), in which an observed event causes neurons to fire more frequently making the experience intense and increasing the likelihood that an event is encoded as a memory. Emotion tends to increase attention, and the emotional element of an event is processed unconsciously in the brain leading to the amygdala.

The inputs are decoded within the respective sensory cortexes, and then these inputs are combined in the hippocampus into a single experience. The hippocampus is then analyze these inputs and decide if they will be comitted to long-term memory. The parallel threads of information are then stored in different parts of the brain.

Encoding occurs on different levels, first is the formation of shor-term memory from the sensory memory, followed by the conversion to a long-term memory by a process of memory consolidation. The process creates a memory trace in response to the external stimuli in the environment.

The hippocampus, deep within the medial temporal lobe, receives connections from the sensory cortexes, and from associative areas and the rhinal and entorhinal cortexes. While theses anterograde connections coverage at the hippocampus, other retrograde pathways emerges, returning to the sensory cortexes. A neural network of cortical synapses effectively encode the multiple associations which are linked to the unit memory.

Consolidation

The consolidation phase is the process of stabilizing a memory trace after the acquisition phase. It consist of two specific processes, synaptic consolidation (occurs after few hours after learning) and system consolidation (where hippocampus-dependent memories become independent of the hippocampus over a period of weeks to years)

The consolidation process generates long-term potentiation, which allows a synapse to increase the connection strength and frequency of signal transmission between two neurons. Potentiation is the process by which synchronous firing of neurons makes those neurons more likely to fire together in the future. Long-term potentiation occurs when the same cluster of neurons fire together to reinforce the connection of the neural circuit. As new experiences is acquired, the brain creates more connections and pathways, and it may re-wire itself by rerouting connection and re-arranging its topological organization.

The ability of the connection or synapse between two neurons to change its connection strength is known as synaptic plasticity or neural plasticity, the brain organizes and reorganizes itself in responses to experiences, creatring new memories.

When the clusters of neural circuits is traversed multiple times, robust connection is made and the likelihood of the same pattern of neural signals streams to flow in the neural pathways of least resistance is increased. This process is achieved by the production of new proteins to rebuild the synapses in the new shape, without which the memory remains fragile and easily decayed over time.

Each neuron makes thousands of connections with other neurons, memories and neural connections are mutually interconnected in extremely complex ways. Each memory is embedded in many connections, and each connection is involved in several memories. Thus, multiple memories may be encoded within a single neural network, by different patterns of synaptic connections. Conversely, a single memory may involve simultaneously activationg several different clusters of neural circuits in completely different parts of the brain.

High quality sleep (specificaly REM, the slow-wave deep sleep) is fundamental in improving the consolidation of information in memory. The activation patterns during sleep replays the learning experience rapidly. Effectively causing long-term potentiation to specific neural circuit in order to transfer the learning experience to long-term memory.

Memory re-consolidation is the process of previously consolidated memories being recalled and then actively consolidated all over again, in order to maintain, strengthen and modify memories that are already stored in the long-term memory. Several retrievals of memory (either naturally through reflection, or through deliberate recall) may be needed for long-term memories to last for many years, depending on the depth of the initial processing.

The act of re-consolidation, may change the intial memory. As a particular memory trace is reactivated, the strengths of the neural connections may change, the memory may become associated with new emotional or environmental conditions or subsequently acquired knowledge.

There are four main types of encoding: - Acoustic encoding is the processing of sound, words and other auditory input for storage and later retrieval. This is a parallel to the concept of the phonological loop which allows input within our echoic memory to be sub-vocally rehearsed in order to remember the sound sequences. - Visual encoding is the processing of images and visual sensory information. Visual input is temporarily stored within the iconic memory before being encoded into long-term memory. The amygdala (within the medial temporal lobe of the brain which has a primary role in the processing of emotional reactions) play an important role in visual encoding, as it accepts visual input in addition to input from other systems and encodes the positive or negative values of conditioned stimuli. This is also has a parallel to the concept of visuo-sketchpad which allows manipulation of mental imagery. - Tactile encoding is the processing of how something feels through the sense of touch. Physiologically, neurons in the somatosensory cortex of the brain react to vibrotactile stimuli caused by the feel of an object. - Semantic encoding is the processing of sensory input that has particular meaning or particular context, instead of deriving from a particular sense. The encoding of meaning to facts and events allows creation of more abstract memory which are linked to more specific concepts.

Memory is fundamentally associative, meaning that a new piece of information is remembered better if it can be associated with previously acquired knowledge that is already stored in memory. The more personally meaningful the association, the more effective the encoding and consolidation. Elaborate processing that emphasizes meaning and associations that are familiar leads to improved recall. Otherwise, information that is difficult to understand and associate cannot be readily integrated with already acquired knowledge, and it will be poorly remembered, and may even be remembered in a distorted form due to the effort to comprehend its meaning and associations. It means that the process of understanding an idea and concepts on the deeper level avoids the formation of distorted or false memory because of deeper and clear understanding.

Storage

Storage is a passive proess of retaining information in the brain, whether in the sensory memory, the short-term memory or the long-term memory. Each of these different stages of human memory function as a sort of filter that helps to protect us from the stream of information that confront us on a daily basis, avoiding an overload of information and helping to keep us sane. The more the information is repeated or used, the more likely it is to be retained in long-term memory.

Long-term memories are not stored in just one part of the brain, but are widely distributed throughout the cortex. After consolidation, long-term memories are stored throughout the brain as cluster of neurons that are primed to fire together in the same pattern that created the original experience, and each component of a memory is stored in the brain area that initiated it. It seems that they may even be encoded redundantly, several times, in various parts of the cortex, so that, if one engram (or memory trace) is wiped out, there are duplicates, or alternative pathways, elsewhere, through which the memory may still be retrieved.

The human brain has the capacity to store almost unlimited amounts of information indefinitely. Forgetting, therefore, is more likely to be result from incorrectly or incompletely encoded memories, and/or problems with the retrieval process. It is a common experience that we may try to remember something one time and fail, but then remember that same item later. The information is therefore clearly still there in storage, but there may have been some kind of a mismatch between retrieval cues and the original encoding of the information.

Forgetting, is perhaps better thought of as the temporary or permanent inability to retrieve a piece of information or a memory that had previously been encoded in the brain. Forgetting typically follows a logarithmic curve, so that information loss is quite rapid at the start, but becomes slower as time goes on.

Retrieval

Recall or retrieval of memory refers to the re-access of events or information from the past previously encoded and stored in the brain. It is known as remembering. During recall, the brain “replays” a pattern of neural activity that was originally generated in response to a particular event. These replays are not identiacal to the original neural activity. Thus, remembering can be thought of as an act of creative reimagination of the past experience.

Because of the way memories are encoded and stored, memory recall is effectively an on-the-fly reconstruction of elements throughout various areas of our brains. Memories are not stored in our brains like books on library shelves, or even as a collection of self-contained recordings or pictures or videos, but as a kind of a jigsaw puzzle, involving different elements stored in different parts of the brain linked together by associations within the neural networks. Memory retrieval requires replay of the neural pathways in the brain formed when encoding of the memory occured, and the strength of those pathways determines how quickly the memory can be recalled. Recall effectively returns a memory from long-term storage to short-term or working memory, where it can be accessed, as a experience replay of the encoding process. It is then re-stored back in long-term memory, thus re-consolidating and strengthening the memory.

The efficiency of human memory recall is astounding. Most of what we remember is by direct retrieval, where items of information are linked directly to a question or cue. Other memories are retrieved quickly and efficiently by hierarchical inference, where a specific question is linked to a class, categories or subset of information about which certain facts are known. Also, the brain is usually able to determine in advance whether there is any point in searching memory for a particular fact.

There are two main methods of accessing memory: recognition and recall. Recognition is the association of an event or physical object with one previously experience, and involves a process of comparison of information with memory, e.g. recognizing a known face, true/false or multiple choice questions, etc. Recognition is a largely unconscious process, and the brain even has a dedicated face-recognition area, which passes information directly through the limbic areas to generate a sense of familiarity, before linking up with the cortical path, where data about the person’s movements and intentions are processed. Recall involves remembering a fact, event or object that is not currently physically present (in the sense of retrieving a representation, mental image or concept), and requires the direct uncovering of information from memory, e.g. remembering the name of a recognized person, fill-in the blank questions, etc.

Recognition is considered to be easier to recall, in that it requires just a single process rather than two processes. Recognition requires only a simple familiarity decision, whereas a full recall of contents from memory requires a two-stage process in which the search and retrieval of possible contents from memory is followed by a familiarity decision where the correct information is chosen from the possible contents retrieved. Thus, recall involves actively reconstructing the information and requires the activation of all the neural circuit involved in the memory encoding, whereas recognition only requires a relatively simple decision as to whether one thing has been encountered before or not. Sometimes, however, even if a part of an object initially activates only a part of the neural network, recognition may then suffice to activate the entire network.

Memory uses information both from the specific memory trace as well as from the environment in which it is retrieved. Because of its focus on the retrieval environment or state, encoding specificity takes into account context cues. Typically, recall is better when the environments are similar in both the learning (encoding) and recall phases. In the same way, emotional material is remembered more reliably in moods that match the emotional content of these memories (e.g. happy people will remember more happy than sad information, whereas sad people will better remember sad than happy information).

The levels of processing effect refers to memory recall of stimuli to be the function of the depth of mental processing, which is in turn determined by connections with previous memory, time spent processing the stimulus, cognitive effort and sensory input mode. Thus, shallow processing (typically based on sound or writing) leads to a relatively fragile memory trace that is susceptible to rapid decay, whereas deep processing (based on semantics and meanings) results in a more durable memory trace. This means that memory strength is continuously variable.

The evidence suggests that memory retrieval is a more or less automatic process. Although distraction or divided attention at the time of recall tends to slow down the retrieval process to some extent, it typically has little to no effect on the accuracy of retrieved memories. Distraction at the time of encoding, on the other hand, can severely impair subsequent retrieval success.

The efficiency of memory recall can be increased to some extent by making inferences from our personal storage of world knowledge, and by our use of schema (plural: schemata). A schema is an organized mental structure or framework of pre-conceived ideas about the world and how it works, which we can use to make inferences and assumptions about how to interpret and process information. Thus, our everyday communication consists not just of words and their meanings, but also of what is left out and mutually understood. Such schemata are applied to recalled memories, so that we can often flesh out details of a memory from just a skeleton memory of a central event or object. However, the use of schemata may also lead to memory errors as assumed or expected associated events are added that did not actually occur.

There are three main types of recall: - Free recall is the process in which a person is given a list of items to remember and then is asked to recall them in any order (hence the name “free”). This type of recall often displays evidence of either the primacy effect (when the person recalls items presented at the beginning of the list earlier and more often) or the recency effect (when the person recalls items presented at the end of the list earlier and more often), and also of the contiguity effect (the marked tendency for items from neighbouring positions in the list to be recalled successively).

  • Cued recall is the process in which a person is given a list of items to remember and is then tested with the use of cues or guides. When cues are provided to a person, they tend to remember items on the list that they did not originally recall without a cue, and which were thought to be lost to memory. This can also take the form of stimulus-response recall, as when words, pictures and numbers are presented together in a pair, and the resulting associations between the two items cues the recall of the second item in the pair.

  • Serial recall refers to our ability to recall items or events in the order in which they occurred, whether chronological events in our autobiographical memories, or the order of the different parts of a sentence (or phonemes in a word) in order to make sense of them. Serial recall in long-term memory appears to differ from serial recall in short-term memory, in that a sequence in long-term memory is represented in memory as a whole, rather than as a series of discrete items. Testing of serial recall by psychologists have yielded several general rules:

    • more recent events are more easily remembered in order (especially with auditory stimuli);
    • recall decreases as the length of the list or sequence increases;
    • there is a tendency to remember the correct items, but in the wrong order;
    • where errors are made, there is a tendency to respond with an item that resembles the original item in some way (e.g. “dog” instead of “fog”, or perhaps an item physically close to the original item);
    • repetition errors do occur, but they are relatively rare;
    • if an item is recalled earlier in the list than it should be, the missed item tends to be inserted immediately after it;
    • if an item from a previous trial is recalled in a current trial, it is likely to be recalled at its position from the original trial.

If we assume that the “purpose” of human memory is to use past events to guide future actions, then keeping a perfect and complete record of every past event is not necessarily a useful or efficient way of achieving this. So, in most people, some specific memories may be given up or converted into general knowledge (i.e. converted from episodic to semantic memories) as part of the ongoing recall/re-consolidation process, so that that we are able to generalize from experience.

It is also possible that false memories (or at least wrongly interpreted memories) may be created during recall, and carried forward thereafter. Research into false memory creation is particularly associated with Elizabeth Loftus’ work in the 1970s. Among many other experiments in this area (see the side panel on the Psychogenic Amnesia page, for example), she showed how the precise wording of a question about memories (e.g. “the car hit” or “the car smashed into”) can dramatically influence the recall and re-creation of memories, and can even permanently change those memories for future recalls - a phenomenon which is not lost on the legal profession. It is thought that it may even be possible, up to a point, to choose to forget, by blocking out unwanted memories during recall, a process achieved by frontal lobe activity, which inhibits the laying down or re-consolidation of a memory.

Language

Semantic Network
    Node is a Concept
    Link is a Connection

Attention

Selective Attention
Capacity

Problem Solving

Trial-and-Error
Algorithm
Heuristic

Decision Making

Trade-off
Time
Information
Frame of Reference

Meta-Cognition

Meta-Learning

Motivation

Determination

Learning Optimization

Learning problem formulation

  • Learning = Language + Memory + Attention + Time + Energy + Complexity
  • Learning = Representation + Evaluation + Optimization

  • Learning = Information Processing/Computation [Computational Complexity, Time Complexity, Space Complexity]

  • Objective Function

    • Maximize long-term memory information retention, minimize error and decay
    • optimal encoding to retain long-term memory

  • Constraints

    • Working Memory Capacity
    • Attention
    • Time
    • Energy
    • Complexity

  • Learning

    • Encode Memory
    • Store Memory
    • Retrieve Memory