Fractal Brain Resiliency and Adaptability

Highlighted below are the resiliency-related unique features of Fractal Brain reinforcement learning that set it apart from its closest competitors, the policy and value gradient Deep RL methods. Specifically, what derails the policy/value gradient methods are domains that exhibit domain drift, a situation wherein the distribution that generates domain data changes, at test time. This is because the underlying NN parametric models of DeepRL algorithms do not change at test time and therefore have no capacity to adapt themselves to domain changes. In contrast, Fractal Brain keeps rewiring itself, at test time, capturing and adapting itself to any unforeseen domain changes, to remain competitive.

To illustrate this domain adaptation of FractalBrain on a simple example, consider a domain below where a random agent policy scores approx. 10 in a decision epoch whereas a trained agent scores approx. 40. When the underlying domain model is inverted (+ive rewards become -ive) around decision epoch 10k, the random agent performance drops to -10. In contrast, FractalBrain performance does not invert to -40; Instead, the algorithm detects the model drift, adapts itself to it, recomputes the new optimal policy and keeps following it, managing to maintain a strong positive delta in performance over the random policy.

The continual learning and network rewiring ability of Fractal Brain also gives rise to the system resiliency to physical damage and hardware fault-tolerance, arguably a desired feature c. 2024, when TSMC yield of its 3nm process hovers around 55% mark. To illustrate this feature on an example below, consider a trained Fractal Brain RL agent whose performance oscillates around reward=30 mark in each decision epoch. When in decision epoch 7k FractalBrain network is subjected to a 50% (or even 75%) damage of its randomly chosen neurons, the system performance immediately drops. Notice though that Fractal Brain eventually manages to recover its original performance, effectively learning to leverage the still operational parts of its network to encode the policies that originally resided in the physically damaged parts of network.

Continual expansion of Fractal Brain connectome is clearly advantageous when it comes to the overall system resiliency and adaptability, but what about its growing memory footprint? To address this issue, similarly to modern computing languages, Fractal Brain employs garbage collection facility that keeps the overall system memory footprint in check. Specifically, when the system memory usage is considered to be too high, garbage collector distills the connectome by removing from it the least used parts (made of compute units and edges connecting them) and later recycling these parts, to be reused for encoding new Fractal Brain patterns.

The example of this process is best illustrated in the following Figure. Here, the system is trained auto-regressively on an excerpt from the PILE dataset to predict the next word in a sequence. As can be seen in the Figure, Fractal Brain learned to perform well on that task, managing to drive down the predicted word error rate (wer_error) to 0%. Notice, that once the maximum number of units and edges hit their maximum allowed values (600k and 400k respectively), garbage collector kicked-in, working aggressively ever after to ensure that the network memory footprint remains bounded.