Anatomy of a Run Report

A Colimit run report is an AI-enhanced version of the GitHub Actions Workflow Job run that triggered it, which acts as a Debugging Workbench to help you debug the failed build.

Colimit generates these automatically whenever one of your GitHub Actions Workflow Job runs fails.

A run report is made up of the following parts, which are summarized below:

Workflow Job Metadata

Basic info about the commit and branch from which the failure originated, the GitHub user that triggered the run, when it was created and how long it took, and a link to the raw run on GitHub.

Command Output

A filtered version of the run logs, which plucks out only the critical errors that caused the run to fail. If you've ever had to debug a CI run on GitHub, you know how much squinting you normally have to do to ignore the noise and find the actual errors.

Relevant Files

Colimit agent crawls your indexed codebase, looking for files that are relevant to debugging the failures in your build logs.

This goes beyond just looking at modified files or transitive dependencies, it semantically considers what your code does when finding the files necessary to debug the failure.

These files become the additional context that Colimit uses to debug the failure and generate a run report.

Debugging Notes

The debugging notes consist of two major tabs, a Solution and an Analysis.

Solution Tab

The Solution tab contains the most likely cause of the bug, how to fix it, and a diff of the changes for the fix. You can Copy Changes to you clipboard to try it locally, or commit directly to your branch from Colimit.

Analysis Tab

If the bug is particularly difficult, chances are that the Solution won't be correct on the first try. This is where the Analysis tab comes in, which contains a Root Cause Analysis report consisting of:

• Important Log Excerpts and explanations about why they're important
• Top 3 Potential Causes of the bug (including supporting evidence, contradictory evidence, and a confidence score)
• Top 3 Fix Options for the most likely cause (including pros and cons for each, and codebase implications)

You're then able to Add Context about what you've already considered to interact with the analysis report, functioning as a debugging workbench, and allowing you to optionally use a more thorough Deep analysis mode for hard bugs, or a Quick analysis mode if you just want minor changes to the original fix.