Graph data analysis have recently attained much interests from the statistics community for their capabilities to perform inferential tasks on types of data that are prevailing at every corner of our daily life. Another hot topic is neural network which has been applied on various disciplines, like partial deferential equation(PDE), for their capabilities to deal with high dimensional problems that traditional finite elements or finite difference methods often fail to solve. My dissertation develops several methods that focus on various aspects across these two disciplines. In the graph-related realm, we firstly developed efficient numerical routines for unsupervised clustering problem and further proved theoretical guarantees on the clustering error corresponding to the numerical routines. For graph neural network, we modified the traditional training process of graph neural network by re-sampling the graph structure. To the best of our knowledge, we are the first to introduce the randomness into the training phase of graph neural network. On the PDE domain, we proposed structure-preserving neural-network-based numerical schemes to solve both L2-gradient flows and generalized diffusions. By using neural networks as tools for spatial discretization, we introduce a structure-preserving Eulerian algorithm to solve L2-gradient flows and a structure-preserving Lagrangian algorithm to solve generalized diffusions.