Sankalp Garg

A method discloses receiving a plurality of input images, receiving text prompts, generating a visual matrix utilizing the images and an image encoder, generating a text matrix utilizing a text encoder, multiplying the text matrix and the visual matrix to generate an image-text similarity matrix that assigns a numerical value indicating similarities between each of encoded visual descriptors and each of the encoded images, wherein similarities are indicated by entries of the image-text similarity matrix having numerical values that determine a loss function associated with the image-text similarity matrix, identify a gradient of the loss function with respect to parameters associated with the image encoder and parameters associated with the text encoder, utilizing the gradient, update parameters associated with the image encoder or the text encoder, and outputting final updated parameters associated with either the text encoder or image encoder of the machine learning network.

This work shows that a natural and simple approach of mimicking contrastive pretraining consistently outperforms alternative finetuning approaches, and establishes the proposed method of contrastive finetuning as a simple and intuitive state-of-the-art for supervised finetuned of image-text models like CLIP.

Presents SymNet, the first neural approach for solving Relational MDPs expressed in RDDL, demonstrating that SymNet policies are significantly better than random and sometimes more effective than training state-of-the-art deep reactive policies from scratch.

This paper proposes a new framework, DArtNet, which learns a static embedding for every node in the graph as well as a dynamic embedding which is dependent on the dynamic attribute value (time-series) and captures the information from the neighborhood by taking a relation specific mean and encodes the history information using RNN.

This work presents the first method for neural transfer of RDDL MDPs that can transfer across problems of different sizes and has superior learning curves over training from scratch.

A Deep Neural Network (DNN) for fall detection is proposed and proved to be independent of filtering operations suggesting the approach to be useful in noisy environment.

This paper presents the first domain-independent transfer algorithm for MDP planning domains expressed in an RDDL representation, which exploits the symbolic state configuration and transition function of the domain to learn a shared embedding space for states and state-action pairs for all problem instances of a domain.