Toolkit Run History

​

Overview

The Run History panel serves as a valuable auditing and troubleshooting tool, allowing teams to:

• Monitor toolkit usage patterns and frequency
• Review past executions with full input/output details
• Troubleshoot failures by examining error messages
• Verify successful operations and their results
• Track toolkit performance over time

---

​

Access Run History

1. Open the toolkit detail page.
2. Select Run History in the top-right corner of the left panel.
3. The dedicated Run History page for this toolkit opens. To return, use the breadcrumb at the top of the page.

---

​

Run History View

The Run History page is a dedicated full-page view. At the top, it shows a breadcrumb—Toolkits & Indexes / [toolkit name] / Run History—where the “Toolkits & Indexes” and toolkit name links are clickable for navigation back. Below the breadcrumb is a two-panel layout: a left panel listing all past runs, and a right panel showing the selected conversation replay.

​

Left Panel

The left panel displays a sortable list of all toolkit runs with the following columns: Features:

• Sorting: Select any column header to toggle sort order (ascending ↑ or descending ↓). Default: Date descending (most recent first)
• Selection: Select any row to view details in the right panel. The selected row is highlighted
• Infinite scroll: Scroll to the bottom to automatically load more entries
• Three-dot menu (⋮): Each row has a context menu available on hover with run actions

​

Right Panel

The right panel displays comprehensive execution details for the selected run in read-only format. You cannot interact with or modify the displayed conversation. Displayed Information:

• User Input: Original queries or commands
• Tool Execution: Tool names and parameters passed
• Tool Output: Results and data returned
• Error Messages: Detailed error information with stack traces
• Timestamps: Timing information for each step

​

Run Actions

Each run row includes a three-dot menu (⋮) that appears on hover, with these actions:

---

​

View Run Details

Open Run History as described in Access Run History, then select any row in the left panel. The right panel loads the preserved conversation in read-only format, where you can scroll through the full execution flow, including the toolkit tools used in that run, the parameters passed to each tool, the returned results, and any errors with stack traces.

---

​

Common Use Cases

Debug a Failed Run

Scenario: A toolkit operation produced unexpected results or failed to complete.Steps:

1. Open the Run History page.
2. Find the failed run by date.
3. Select the run to view the execution details.
4. Review the details to identify:
   • What input triggered the issue
   • Which tool or tools failed
   • Any error messages or stack traces
   • Parameters that were passed
5. Use this information to fix the toolkit configuration or input parameters.

Optimize Performance

Scenario: You want to optimize your toolkit’s execution time.Steps:

1. Open the Run History page.
2. Review the Duration column across multiple runs.
3. Select the Duration column header to sort by execution time.
4. Select the longest-running runs to see what operations were performed.
5. Optimize the toolkit based on the longest-running operations.

Verify Toolkit Changes

Scenario: You updated your toolkit configuration and want to verify improvements.Steps:

1. Make changes to your toolkit configuration.
2. Run the same test inputs before and after the changes using Test Settings.
3. Open the Run History page.
4. Compare runs before and after the changes.
5. Verify that the new configuration produces better results.

Audit & Compliance

Scenario: You must provide evidence of what your toolkit processed.Steps:

1. Open the Run History page.
2. Find the relevant run by date.
3. Review the complete execution details.
4. Use Share link from the ⋮ menu to save a direct URL to the run for documentation.

Learn & Train

Scenario: You want to understand how your toolkit handles different inputs.Steps:

1. Open the Run History page.
2. Review multiple runs in the list.
3. Study patterns in successful executions.
4. Identify common failure scenarios.
5. Use these insights to improve your toolkit’s configuration or documentation.

---

​

Best Practices

Regular Review

• Check history periodically: Select the Run History button in the top-right of the left panel regularly to review toolkit runs and catch issues early.
• Monitor trends: Track execution duration and frequency patterns over time.
• Performance baseline: Establish expected duration ranges for different toolkit operations.

Debugging Workflow

1. Reproduce issues: When a problem is reported, select the Run History button and find the specific run first.
2. Analyze context: Review input parameters and error messages in detail.
3. Test fixes: After fixing, execute the toolkit and compare results in Run History.
4. Document findings: Use Share link from the ⋮ menu to save a direct URL to key runs as documentation.

Data Management

• Clean old runs: Periodically delete runs that are no longer needed.
• Keep important runs: Do not delete runs that serve as examples or reference cases.
• Share critical runs: Use the copy link feature to share important findings with your team.

Performance Monitoring

• Track duration trends: Monitor if toolkit operations are getting slower over time.
• Identify bottlenecks: Use duration data to identify which tools need optimization.
• Compare executions: Review multiple runs to understand performance variations.

Security Considerations

• Sensitive data: Be aware that all input parameters and results are stored in run history.
• Access control: Ensure only authorized users can access toolkits with sensitive data.
• Retention policy: Consider establishing a policy for how long to retain run history.

---

​

Troubleshooting

Run History Panel Shows No Entries

Possible Causes:

• No toolkit executions have been performed yet
• All previous runs have been deleted
• You are viewing a newly created toolkit

Solution:

1. Execute the toolkit from the Test Settings area (right panel on the Configuration page).
2. Go back to the toolkit detail page and re-open Run History to refresh.
3. If runs were deleted, they cannot be restored in the UI.

Cannot See Run Details in Right Panel

Possible Causes:

• No run is selected
• Run data failed to load due to network issues
• Run details are still loading

Solution:

1. Select a run in the left panel.
2. Wait a moment for details to load.
3. If details do not appear, go back and re-open Run History.
4. Check your network connection.
5. Select a different run to verify functionality.

Copy Link Feature Not Working

Possible Causes:

• Clipboard permissions not granted
• Browser security restrictions

Solution:

1. Ensure your browser has clipboard access permissions.
2. Hover over the run row to reveal the ⋮ (three-dot menu) icon.
3. Select the icon and select Share link.
4. Check for the success toast notification confirming the link was copied.
5. If using private or incognito mode, check browser clipboard permissions.

Delete Option Not Appearing

Possible Causes:

• Not hovering over the run row
• Insufficient permissions
• UI not fully loaded

Solution:

1. Hover your mouse over the run row to reveal the ⋮ (three-dot menu) icon.
2. Wait briefly for the icon to appear on the right side of the row.
3. Select the icon and look for Delete in the menu.
4. Verify you have the necessary permissions to delete runs.
5. Refresh the page if the menu does not appear.

Runs Not Sorted Correctly

Possible Causes:

• Unexpected sort direction after multiple selections
• Cached data display issue

Solution:

1. Select the column header once to sort ascending.
2. Select again to sort descending.
3. Look for the arrow indicator (↑/↓) showing the current sort direction.
4. Go back and re-open Run History if sorting appears broken.
5. Sort by a different column first, then switch back.

---