Skip to main content
The Prompt Library is a central repository for reusable prompts. It enables organized and repeatable testing of your AI applications.

​
Building a Prompt Library

  1. Go to the Prompt Library tab in the navigation bar.
  2. Click New Prompt Library.
  3. Provide a unique name that reflects the library’s purpose.
    • Example: Customer Service Chatbot Prompts
    • Example: Medical Diagnosis Prompts
1

Create New Prompt

Within your newly created prompt library, you can add individual prompts for evaluation. Trusys currently supports three types of prompts.

  • Text
  • JSON
  • Conversational
Standard text inputs
  • Example: "What is the capital of France?"
  • Example with variables: "Translate the following sentence into French: {{sentence_to_translate}}"
You can use variables within your prompts using the syntax {{var}}. This allows you to create dynamic prompts where different values can be substituted during evaluation, enabling a single prompt to generate multiple test cases. You can use multiple variables per prompt.
2

Define Variables & Metrics

  1. Variables – Define multiple values for variables. Trusys generates test cases for each value.
Example: “Summarize: {{document_text}}” with document_text = [Article A, Article B, Article C] → 3 test cases.
Optionally, link a dataset to populate values (see Mapping Prompt Variables with Dataset).
  1. Metrics – Select evaluation metrics and expected values.
Example: Translation prompt → Fluency: High
Example: Factual question → Accuracy: Correct
Once all the details are filled, you can save the prompt. You can define multiple prompts within a single prompt set, allowing for organized testing of related functionalities.
3

Define Global Metrics

Instead of defining metrics for each individual prompt, you have the option to define global metrics for the entire prompt library. When global metrics are defined, all prompts within that library will automatically use these metrics for measurement, streamlining the configuration process, especially for large prompt libraries with consistent evaluation criteria.

​
Prompt Library List

The Prompt Library List provides an overview of all prompt libraries created within your project. For each library, you can view:
  • The number of prompts it contains.
  • The number of test runs that were executed using that specific prompt library.

​
Prompt Library Details

Clicking on any prompt library from the list will open its Prompt Library Details page. Here, you can:
  • View existing prompts and their metrics: Review all the prompts within the library and the specific metrics configured for each.
  • Edit the library: You have the flexibility to add new prompts, modify existing ones, or delete prompts that are no longer needed.Add, edit, or delete prompts
  • Review all functional evaluations done using this library: A list of all past functional evaluation test runs conducted using the current prompt library is displayed. Clicking on any item in this list will lead you to the detailed results of that specific test run, providing in-depth insights into the AI’s performance during that evaluation.