Frequency Principle, or F-Principle for short, is an important concept in the field of deep learning. It describes the tendency of deep neural networks (DNNs) to fit the target function from low frequency to high frequency during training. This principle was proposed by Shanghai Jiao Tong University […]

Parameter aggregation describes the phenomenon that during the neural network training process, model parameters tend to gather towards specific values or directions.

Cyclomatic complexity is a software metric used to measure the complexity of a program.

The core idea of Dropout is to randomly discard (i.e. temporarily remove) some neurons in the network and their connections during the training process to prevent the model from overfitting.

Graph Attention Networks (GATs) are a type of neural network designed for graph-structured data. They were proposed by Petar Veličković and his colleagues in 2017. The related paper is “Graph Attention Networks (GATs)”.

Message Passing Neural Networks (MPNN) is a neural network framework for processing graph structured data. It was proposed by Gilmer et al. in 2017. The related paper is “Neural Messa […]

Graph Convolutional Networks (GCN), Kipf and Welling published a paper titled “Semi-Supervised Classification” at the 2017 ICLR conference.

The Gated Recurrent Unit (GRU) is a variant of the Recurrent Neural Network (RNN) proposed by Cho et al. in 2014. The related paper is “Empirical Evaluation of Gate […]

AlexNet is a deep convolutional neural network (CNN) proposed by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton in 2012 and used in the ImageNet image classification competition that year.

CART Decision Tree is a decision tree algorithm that can be used for classification and regression tasks.

Gradient Boosting is an ensemble learning algorithm that builds a strong prediction model by combining multiple weak prediction models (usually decision trees).

LeNet-5 is a pioneering work in the field of deep learning and convolutional neural networks, which laid the foundation for many key concepts in modern deep learning, such as convolutional layers, pooling layers, and fully connected layers.

Qualification questions focus on how to determine all the conditions or factors required for an action or event to be successfully performed in a changing environment.

ReAct proposes a general paradigm that combines advances in reasoning and action to enable language models to solve a variety of language reasoning and decision-making tasks.

Pre-training Once is a three-branch self-supervised training framework that introduces elastic student branches and randomly samples sub-networks for training in each pre-training step.

FlexAttention is a flexible attention mechanism designed to improve the efficiency of high-resolution vision-language models.

FlashAttention is an efficient and memory-friendly attention algorithm.

Causal Attention (CATT) is an innovative attention mechanism that improves the interpretability and performance of models by incorporating causal inference, especially in vision-language tasks. This mechanism was first proposed by researchers from Nanyang Technological University and Monash University in Australia in 20 […]

Thought Trees generalize the popular thought chaining approach to prompt language models and enable the exploration of coherent text units (thoughts) as intermediate steps in problem solving.

The MoMa architecture is a novel modality-aware mixture of experts (MoE) architecture designed for pre-training mixed-modality, early-fusion language models.

Multi-step Error Minimization (MEM) was published in 2024 by the Institute of Information Engineering of the Chinese Academy of Sciences, Nanyang Technological University, National University of Singapore, and Sun Yat-sen University in the paper “Multimodal Unlearnable E […]

The Geometric Langlands Conjecture is a geometric version of the Langlands program.

The Langlands Program is a highly influential research field in modern mathematics. It involves multiple branches of mathematics such as number theory, algebraic geometry and group representation theory, and attempts to reveal the profound connections between them.

An application-specific integrated circuit (ASIC) is an integrated circuit designed and manufactured according to specific user requirements and the needs of a specific electronic system.

Frequency Principle, or F-Principle for short, is an important concept in the field of deep learning. It describes the tendency of deep neural networks (DNNs) to fit the target function from low frequency to high frequency during training. This principle was proposed by Shanghai Jiao Tong University […]

Parameter aggregation describes the phenomenon that during the neural network training process, model parameters tend to gather towards specific values or directions.

Cyclomatic complexity is a software metric used to measure the complexity of a program.

The core idea of Dropout is to randomly discard (i.e. temporarily remove) some neurons in the network and their connections during the training process to prevent the model from overfitting.

Graph Attention Networks (GATs) are a type of neural network designed for graph-structured data. They were proposed by Petar Veličković and his colleagues in 2017. The related paper is “Graph Attention Networks (GATs)”.

Message Passing Neural Networks (MPNN) is a neural network framework for processing graph structured data. It was proposed by Gilmer et al. in 2017. The related paper is “Neural Messa […]

Graph Convolutional Networks (GCN), Kipf and Welling published a paper titled “Semi-Supervised Classification” at the 2017 ICLR conference.

The Gated Recurrent Unit (GRU) is a variant of the Recurrent Neural Network (RNN) proposed by Cho et al. in 2014. The related paper is “Empirical Evaluation of Gate […]

AlexNet is a deep convolutional neural network (CNN) proposed by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton in 2012 and used in the ImageNet image classification competition that year.

CART Decision Tree is a decision tree algorithm that can be used for classification and regression tasks.

Gradient Boosting is an ensemble learning algorithm that builds a strong prediction model by combining multiple weak prediction models (usually decision trees).

LeNet-5 is a pioneering work in the field of deep learning and convolutional neural networks, which laid the foundation for many key concepts in modern deep learning, such as convolutional layers, pooling layers, and fully connected layers.

Qualification questions focus on how to determine all the conditions or factors required for an action or event to be successfully performed in a changing environment.

ReAct proposes a general paradigm that combines advances in reasoning and action to enable language models to solve a variety of language reasoning and decision-making tasks.

Pre-training Once is a three-branch self-supervised training framework that introduces elastic student branches and randomly samples sub-networks for training in each pre-training step.

FlexAttention is a flexible attention mechanism designed to improve the efficiency of high-resolution vision-language models.

FlashAttention is an efficient and memory-friendly attention algorithm.

Causal Attention (CATT) is an innovative attention mechanism that improves the interpretability and performance of models by incorporating causal inference, especially in vision-language tasks. This mechanism was first proposed by researchers from Nanyang Technological University and Monash University in Australia in 20 […]

Thought Trees generalize the popular thought chaining approach to prompt language models and enable the exploration of coherent text units (thoughts) as intermediate steps in problem solving.

The MoMa architecture is a novel modality-aware mixture of experts (MoE) architecture designed for pre-training mixed-modality, early-fusion language models.

Multi-step Error Minimization (MEM) was published in 2024 by the Institute of Information Engineering of the Chinese Academy of Sciences, Nanyang Technological University, National University of Singapore, and Sun Yat-sen University in the paper “Multimodal Unlearnable E […]

The Geometric Langlands Conjecture is a geometric version of the Langlands program.

The Langlands Program is a highly influential research field in modern mathematics. It involves multiple branches of mathematics such as number theory, algebraic geometry and group representation theory, and attempts to reveal the profound connections between them.

An application-specific integrated circuit (ASIC) is an integrated circuit designed and manufactured according to specific user requirements and the needs of a specific electronic system.