Here’s an interesting talk from Microsoft Research YouTube channel by Yuija Li about Gated Graph Sequence Neural Networks. Details about the presentation and a link to the paper are below the video.

Link to paper

From the description:

Graph-structured data appears frequently in domains including chemistry, natural language semantics, social networks, and knowledge bases. In this work, we study feature learning techniques for graph-structured inputs. Our starting point is previous work on Graph Neural Networks (Scarselli et al., 2009), which we modify to use gated recurrent units and modern optimization techniques and then extend to output sequences. The result is a flexible and broadly useful class of neural network models that has favorable inductive biases relative to purely sequence-based models (e.g., LSTMs) when the problem is graph-structured. We demonstrate the capabilities on some simple AI (bAbI) and graph algorithm learning tasks. We then show it achieves state-of-the-art performance on a problem from program verification, in which subgraphs need to be matched to abstract data structures.

Interpreting what neural networks are doing is a tricky problem. In fact, they are often referred to as a “black box.”

In this video Arxiv dives into the approach of feature visualization. From simple neuron excitation to the Deep Visualization Toolbox and the Google DeepDream project.

Watch to open up that black box!

While at NDC in Sydney, Carl and Richard talked to Joe Albahari about using LINQPad to create neural nets from scratch.


LINQPad is an interactive development environment for .NET – originally focused on helping you build LINQ expressions. But as Joe explains, it can be used for all sorts of interactive coding experiences – including learning to build neural networks. Joe talks through the fundamentals of neural nets and what it’s like to build neural nets yourself. Even if you move on to more advanced machine learning tooling, learning the fundamentals are useful!

Joe Albahari is an O’Reilly author and the inventor of LINQPad. He’s written seven books on C# and LINQ, including the upcoming “C# 7.0 in a Nutshell”. He speaks regularly at conferences and user groups, and has been a C# MVP for nine years running.

Press the play button below to listen here or visit the show page.