Vector Search and Filtering

Once a user query has been embedded into its semantic vector representation, DAKSH initiates a top-k vector similarity search within its high-performance vector database. This search process identifies the most contextually relevant knowledge chunks from the indexed corpus using cosine similarity or inner product scoring, depending on the active configuration and precision requirements. The system typically retrieves 8 to 15 candidate chunks that demonstrate the highest proximity to the query vector in the latent space.

However, raw similarity alone is insufficient for generating precise, trustworthy responses — especially in enterprise and governance contexts where access control, recency, and linguistic fidelity are critical. Therefore, DAKSH applies a multi-layered post-retrieval filtering system to refine the results and ensure that only the most suitable knowledge units are passed on to the generation module.

The first stage of this process is relevance thresholding. Chunks that fall below a minimum similarity threshold — indicating insufficient semantic alignment — are discarded. This helps prevent loosely related or noisy content from diluting the context window.

Next, access-based filtering is enforced. Each chunk carries metadata about its visibility level (e.g., public, internal, admin-only), and DAKSH filters results based on the current user's role, authorization scope, or assigned context. This ensures strict compliance with enterprise data access policies.

Following access control, DAKSH performs recency and authority scoring. In cases where multiple documents contain answers to the same query, the system prioritizes the version with the most recent timestamp and/or the highest authority — such as official policies, verified SOPs, or regulatory circulars. This weighting model helps mitigate outdated or conflicting information from reaching the user.

Finally, the system enforces language matching. Chunks that are not in the same language as the user’s query — and are not reliably translatable — are removed. This preserves semantic fidelity and avoids mismatches in multilingual deployments.

Through this rigorous filtration pipeline, DAKSH typically narrows down the candidate pool to 3–5 highly relevant, role-appropriate, and contextually grounded knowledge chunks, ensuring that the generation engine produces responses that are accurate, compliant, and linguistically consistent.

Updated on