Using a Weaviate RAG Definition
To use your predefined Weaviate RAG Integration:
- In a Stage or in Common Stage Settings for the RAG/Content Retrieval field, select your defined Weaviate RAG Integration and fill out the following fields
| Field | Details |
|---|---|
| Resolve ambiguous language before RAG Retrieval | If set to ON, resolve ambiguous references in user messages before attempting RAG retrieval. For example, during a conversation about the Eiffel Tower, you might wish to resolve ambiguous user queries such as 'Where is it?' - resolving 'it' to a more concrete query eg. to 'Where is the Eiffel Tower located?'. This mechanism uses the recent transcript to help resolve these 'anaphoric' references before attempting to retrieve related passages. |
| Total Prior Messages to Use | Specify the total amount of messages to use when resolving any ambiguities in the user's query before RAG retrieva |
| Maximum Documents to Retrieve (Top K) | Specify the maximum documents to retrieve from the RAG store |
| Maximum Tokens Threshold | Limit the total tokens passed to the LLM from all documents after RAG Retrieval has been performed. This threshold will not truncate messages i.e. the inclusion of full documents may result in more tokens than specified here being passed to the LLM. |
| Distance Threshold | Specify the maximum distance that qualifies documents to be included in the RAG retrieval. You may need to experiment to determine the best value for your data and your agent. |
| Additional Metadata Fields To Expose to LLM | A comma-separated list of the names of any metadata fields which should be included in the context shown to the LLM in addition to the retrieved text. |
| References: Fields To Include | Optional - a comma-separated list of metadata fields. You can get the LLM to generate a reference containing metadata fields as part of its response. For example, your RAG metadata may include a URL to the source document which may be useful to make available to the user in your agent. The reference generated by the LLM is detected, extracted and emitted as a Reference event and made available to clients as JSON event objects for further handling. References can include any combination of data from the RAG result metadata. To enable references, supply a comma-separated list of the names of any metadata fields to include in the reference. Note that you may also need to experiment with reinforcing language referring to references in your prompting to encourage the LLM to generate references reliably eg. 'Ensure that you include any available references if available and appropriate'. |