> For the complete documentation index, see [llms.txt](https://pmse.gitbook.io/pmse-dhdk/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://pmse.gitbook.io/pmse-dhdk/4.-verification-and-validation/4.3-sample-test-suite-structure.md).

# 4.3 Sample Test Suite Structure

The test suite for the GNA chatbot is structured into <mark style="color:blue;">**modules**</mark>, each encompassing <mark style="color:blue;">**test cases**</mark> organized by <mark style="color:blue;">**specific functionality**</mark>. Designed for the prototype version of the application, this test suite aims to validate the core functionalities, ensuring both reliability and efficiency.

{% stepper %}
{% step %}

### **Test Plan**

* **Objectives**: Verify the proper functioning of dataset loading, vector store management, conversational AI integration, and the Streamlit interface.
* **Scope**: Cover all functionalities of the prototype, focusing on integration points, semantic understanding of the AI system and user-facing elements.
* **Timeline**: Include milestones for unit testing, integration testing, and user acceptance testing (UAT).
  {% endstep %}

{% step %}

### **Test Cases**

Each key functionality is covered with test cases:

* **Dataset Loading and Chunking**
  * Validate the dataset's structure and ensure correct loading.
  * Verify chunking logic produces manageable and meaningful content divisions.
  * Confirm metadata enrichment (e.g., title, description, URL) for each chunk.
* **Vector Store Management**
  * Test the creation of a FAISS vector store from a dataset.
  * Validate semantic search functionality for chunk retrieval.
  * Ensure proper loading of the vector store without corruption.
* **Conversational AI Integration**
  * Verify text generation quality from the Mistral NeMo model.
  * Ensure the retrieval of relevant chunks.
  * Validate conversational continuity across multiple user queries.
* **Retry Mechanism for API Calls**
  * Test graceful handling of network issues and API rate limits.
  * Ensure retries do not lead to duplicate or erroneous outputs.
* **Streamlit Interface**
  * Confirm proper rendering of UI elements.
  * Validate conversation history display, including user and  mchatbotessage styling.
  * Test responsiveness and error messages for invalid user inputs.
* **Error Handling and Debugging**

  * Simulate common issues (e.g., missing API keys, invalid dataset) to ensure proper logging and error reporting.

  <mark style="color:blue;">**Testing Tool:**</mark> Pytest library is used for unit and integration tests.
  {% endstep %}

{% step %}

### **Test Data**

* **Dataset Loading:** Use a single CSV file with a specific structure, size, and free of edge cases.
* **Vector Store:** Include a database with unique metadata to test indexing and retrieval accuracy.
* **Chatbot application:** Provide sample queries to test semantic understanding and response generation.

{% hint style="danger" %}
In order to ensure proper alignment with the project's objectives and knowledge retrieval requirements, sample queries must be competency questions (CQs) explicitly defined by the project grantor (i.e., GNA).
{% endhint %}
{% endstep %}

{% step %}

### **Test Environment**

* Configuration includes:
  * Devices: Desktop and laptop systems.
  * Operating Systems: Windows, macOS, and Linux.
  * Network Conditions: Stable and unstable network environments to test retry mechanisms.
* Ensure compatibility with required libraries and tools (e.g., FAISS, Streamlit).
  {% endstep %}

{% step %}

### **Defect Tracking**

* Use <mark style="color:blue;">**Jira**</mark> to log, prioritize, and track issues.
* Categorize defects by severity (e.g., critical, major, minor).
* Include fields for defect description, reproduction steps, and resolution status
  {% endstep %}
  {% endstepper %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://pmse.gitbook.io/pmse-dhdk/4.-verification-and-validation/4.3-sample-test-suite-structure.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.