Application

• Select Model – Pick your LLM from available options
• Authentication – Supply credentials to securely connect

• Provide the Phone Number associated with the AI-powered calling system
• Trusys automatically captures and evaluates conversational interactions

• Flowise
• Langflow
• Dify
• Microsoft Foundry Agents
• CrewAI
• n8n

• Select the agentic framework
• Provide framework-specific authentication details
• Configure connection parameters required by the selected framework

• Monitor multi-agent workflows
• Evaluate agent decisions and outputs
• Track security and functional risks across agent interactions

• Select the Custom Provider Type
• Choose the Request Type (e.g., GET, POST)
• Enter the Base URL of the application’s API.
• Provide API authentication details for secure communication.
• Define how to transform response data for compatibility with Trusys’s systems.

{
"chat_history": [
{% for completion in _conversation %}
{
  "role": "user",
  "content": "{{ completion.input }}"
},
{
  "role": "assistant",
  "content": "{{ completion.output }}"
},
{% endfor %}
{
  "role": "user",
  "content": "{{prompt}}"
}
]
}

​

Using Custom Provider

1. Develop your custom provider script: Write a JavaScript or Python file that implements the callApi function (or call_api for Python) to interact with your AI model or API. This function will recieve the prompt generated by the Trusys platform as an argument. The code should submit this prompt to the AI application and return the output. Please see examples provided below.
2. Select the custom provider: When configuring the application, choose the language in which you have implemented the custom provider.
3. Run the tests using CLI: Run the evaluations using the CLI by providing the path to your custom provider file: Command-Line Usage.

class EchoProvider {
id() {
return 'echo';
    }
async callApi(prompt, context, options) {
return {
  output: `Echo: ${prompt}`,
      };
  }
    }
module.exports = EchoProvider;

def call_api(prompt, options, context):
"""Simple provider that echoes the prompt with a prefix."""
config = options.get("config", {})
prefix = config.get("prefix", "Echo: ")

return {
    "output": f"{prefix}{prompt}"
}