the combination of expert systems or knowledge based systems and inference systems (ML) to produce smaller but more capably systems... best of both worlds - Transformers networks can produce knowledge but requires huge amounts of training and stores it wastefully, spread in the network.
That's an interesting point Ben. There might be some benefit to distributed representations within the larger network which can spark unexpected connections, yielding novelty which we tend to value.
the combination of expert systems or knowledge based systems and inference systems (ML) to produce smaller but more capably systems... best of both worlds - Transformers networks can produce knowledge but requires huge amounts of training and stores it wastefully, spread in the network.
That's an interesting point Ben. There might be some benefit to distributed representations within the larger network which can spark unexpected connections, yielding novelty which we tend to value.