RSTSR 2nd Report: Understanding and Requirements of Electronic Structure Programs: A NumPy+Rust Perspective​

1. Preface​

Development of electronic structure programs aims to solve or discover scientific problems in chemistry and material structure; but the technical challenges involved are often unrelated to chemistry itself. Electronic structure programs need to take into account both 1) development efficiency and reduced communication cost, and 2) program efficiency and resource control; these are also problems that scientific computing in other disciplines, and even general program development tasks, need to solve.

The Rust language has received good reception in some fields of computer science or its applications. But for scientific computing, Rust rarely has outstanding work; and I believe there is still no consensus on whether Rust is suitable for scientific computing.

The previous showcase_rust_riccsd work, I believe, can show that for the problems electronic structure cares about, represented by MP$n$ and CC, it is possible to achieve a good balance between development efficiency and program efficiency with appropriate tools. To demonstrate this possibility, I developed RSTSR as a tensor computing tool. Limited by my horizons, abilities and energy, this tool may not be ideal; but I hope to use this tool to show my understanding of and expectations for Rust scientific computing programs, and to provide some ideas for the development of tensor tools that satisfy electronic structure.

This document hopes to use a simple Q&A style to show my understanding of electronic structure program development and the program problems it cares about, as well as my understanding of the Rust language. The second half of this document will introduce the development ideas of the RSTSR program.

A good program tool should allow users to escape tedious technical details to a certain extent, and focus on their own goals; this is also the original intention of developing math library tools. RSTSR is strongly influenced by NumPy. It is necessary for us to show how NumPy or similar math library tools are concretely used in electronic structure, where their strengths and weaknesses are, and our ideas for improving these problems.

This document assumes that GPU heterogeneity and MPI-scale parallelism are not considered. Some conclusions may not apply to these two situations.

RSTSR: Draft of a Rust High-Dimensional Tensor Data Structure Processing Program Built on the Python Array API​

Abstract​

When writing scientific computing programs, the data storage structure and how to use that data structure for efficient computation are important and fundamental problems that need to be planned early. Today, NumPy and PyTorch have almost become the de facto standards for high-dimensional tensor data structures; the latter also provides a relatively unified interface for heterogeneous programming. But outside Python, except for the LibTorch library in C++, few languages support both high-dimensional tensors and heterogeneous programming. We hope the RSTSR program can resolve this dilemma in the Rust language in the future. The current RSTSR program is based on the Python Array API standard (a subset of the NumPy API), and has correctly implemented some important high-dimensional tensor operations and manipulations, exposing part of the interface in preparation for high-performance CPU computation and GPU computation. But for the goal of reimplementing most computational chemistry concerns on CPU, the current progress still needs 5–10 weeks; CUDA support may need even more time and effort. This document will 1) briefly analyze the current state of existing matrix or tensor libraries; 2) explain the practical significance and goals of the Rust tensor library RSTSR; 3) provide technical details on the features already implemented in RSTSR.