How Are Generative Retrieval and Multi-Vector Dense Retrieval Related To Each Other?

In recent years, there has been a significant surge in the adoption of pre-trained language models, leading to an increase in the use of neural-based retrieval models. One such technique that has gained popularity for its effectiveness is Dense Retrieval (DR), which achieves great ranking performance on a number of benchmarks. The goal of Multi-Vector Dense Retrieval (MVDR) techniques is to use several vectors to describe documents or queries.

In the field of information retrieval, the Generative Retrieval (GR) paradigm shift has recently occurred. In contrast to conventional techniques, Generative Retrieval GR aims to produce suitable document identifiers for a given query immediately. Indexing, retrieval, and rating tasks are handled by a single model that is trained using a sequence-to-sequence architecture. In GR, an encoder-decoder architecture is used to translate queries directly to pertinent document identifiers.

Though its efficacy has been proven, nothing is known about how it interacts with other retrieval techniques, especially dense retrieval models. In a recent study, a team of researchers from Shandong University, China, and the University of Amsterdam has systematically established a connection between state-of-the-art multi-vector dense retrieval and generative retrieval. 

They have discovered similarities between the two methods’ emphasis on semantic matching and training targets. They clarified how the loss function in GR can be rebuilt to resemble the unified MVDR framework by looking at the attention layer and prediction head of the algorithm. They also looked at how GR differs from MVDR in terms of document encoding and alignment.

The team has shared that multi-vector dense retrieval and generative retrieval both use the same framework to determine how relevant a document is to a given query. Both approaches determine relevance by adding the products of the query and document vectors and an alignment matrix.

The team has also examined how generative retrieval makes use of this common foundation, using special techniques to calculate the alignment matrix and document token vectors. They have verified their results with studies and showed that both paradigms have similar phrase matching in their alignment matrices. 

The team has summarized their primary contributions as follows. 

From a Multi-Vector Dense Retrieval (MVDR) perspective, the team has offered fresh insights into Generative Retrieval (GR) and presented a common paradigm for evaluating query-document relevance. 

Study of GR methods: To further improve the comprehension of GR’s implementation, they have explored how it makes use of this framework by looking at special methods for document encoding and alignment matrix computation. 

Analytical Experimentation: A number of in-depth analytical experiments have been carried out using the framework. These experiments have highlighted the term-matching phenomenon and have clarified the properties of different alignment directions in both GR and MVDR paradigms, contributing significantly to the empirical understanding of these retrieval methods.

Check out the Paper and Github. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter. Join our Telegram Channel, Discord Channel, and LinkedIn Group.

If you like our work, you will love our newsletter..

Don’t Forget to join our 39k+ ML SubReddit

The post How Are Generative Retrieval and Multi-Vector Dense Retrieval Related To Each Other? appeared first on MarkTechPost.

Source: Read More