Skip to content

Code Review is reported as successful even when the AI model fails #5630

Description

@rodrigo-itao

Description

I'm using Kilo Code Review integrated with GitHub, where a review is automatically triggered whenever a new Pull Request is opened.

In some cases, the AI model fails during the review process. However, even when the review does not complete successfully, Kilo still reports the GitHub check as successful and displays the message:

Code review completed successfully.

For example, in one of the failed runs, the session log shows the following error:

The model hit its output limit while reasoning and produced no actionable output.

Despite this error, the review status is still marked as Complete in Kilo and succeeded in GitHub.

Expected behavior

If the AI model fails, reaches an output/token limit, or is otherwise unable to produce a valid review, the GitHub check should not be reported as successful.

Ideally, the check should:

  • Be marked as failed or incomplete.
  • Clearly indicate that the review could not be completed.
  • Optionally include the reason for the failure.

Actual behavior

The AI model fails and produces no actionable review, but Kilo reports the review as successfully completed in the Pull Request.

This can be misleading because developers may assume that the PR was successfully reviewed when, in reality, no review was produced.

Additional context

The attached screenshots show an example where the internal session log reports a model failure, while the GitHub integration reports the review as successful.

Image Image

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions