Would be interested in reading your thesis if your willing to link it or if you don't want to dox yourself can you dm me? (No seriously I'm new to lemmy is that a thing?)

Graph Neural Networks are by far the coolest advance in DL architectures and it is also quite interesting that Transformers are simply fully connected Graph Attention Network

I too believe that SSL (and to some extent Unsupervised Learning) is by far the best way to frame learning problems in DL, it has shown to avoid the pesky mode collapse and improves out of distribution inference performance.