"Secret" elmah.io features #4 - Get help from AI and ChatGPT

In this fourth post in the series of "secret" elmah.io features, I want to introduce you to one of several AI features available on elmah.io. We have had machine learning features like automatic bot detection and spike identification for years. But a recent addition to the portfolio of AI features is the integration with ChatGPT to get help solving issues. In this post, I'll show you how to set it up and how it works.

When looking at errors logged in elmah.io (or anywhere else really), it may be easy to spot a fix. A simple null reference exception? Find out why the reference was null or add a null check. But sometimes, fixing errors require hour after hour of trying to replicate errors or even figure out what went wrong. With the development of Large Language Models (LLMs) over the last couple of years, we as developers have been given a huge gift, that will help you resolve errors fast.

Most developers have played with LLMs like ChatGPT or Gemini by now. You have maybe even tried to provide it with a stack trace and some error details to get help resolving errors. That is exactly what the elmah.io integration with ChatGPT does. You either need an API key from OpenAI or an elmah.io Enterprise plan that include AI as part of the subscription. For the rest of this post, I'll use the API key approach.

To set up the ChatGPT integration, go to the Apps tab on the Log Settings page. Here, click the Install button on the ChatGPT app. For this post, I've selected the GPT 3.5 Turbo model from OpenAI and inputted my OpenAI API key:

As already mentioned, there are various models available in the MODEL drop-drown so feel free to play with it. I didn't enable any of the toggles just yet, but we will get to that in a minute.

To log an error in my log, I have created a new ASP.NET Core 8 Razor Pages project, but this could be anything .NET. I have installed the Elmah.Io.AspNetCore NuGet package and configured it as shown in Logging to elmah.io from ASP.NET Core. Finally, I've modified the OnGet method in the Privacy.cshtml.cs file to force an exception:

public void OnGet()
{
    try
    {
        var i = 0;
        var result = 42 / i;
    }
    catch (DivideByZeroException e)
    {
        e.Ship(HttpContext);
    }
}

The Ship method will log any thrown DiviceByZeroException which always happens in the try part of the method. When launching the code, we can inspect the elmah.io UI to see the logged exception:

If you are using the elmah.io UI regularly, everything should look familiar, besides the new AI tab. This will be shown when the ChatGPT app is installed, and we believe that the current error is valid for requesting help from ChatGPT. When selecting the AI tab, I can click the Get suggestion button to get help fixing this exception by ChatGPT:

As you can see in the animation, the steaming output from ChatGPT explains what could be wrong and even starts listing several fixes. This is done by providing ChatGPT with a range of prompts both to warm it up as well as provide details about the error like the stack trace, exception type, and more.

But we can do even better. Let's go back to the Apps tab on Log Settings and edit the settings for the ChatGPT app. I'll enable INCLUDE SOURCE to show how ChatGPT behaves when it also knows the source code causing this error.

Next, I'll install the Elmah.Io.Client.Extensions.SourceCode NuGet package in my ASP.NET Core project. If you are not familiar with this package, I blogged about it in the very first "Secret" elmah.io features post. In short, it will include source code in logged errors by adding the following property to the .csproj file:

<EmbedAllSources>true</EmbedAllSources>

And telling Elmah.Io.AspNetCore to enrich log messages with source:

builder.Services.AddElmahIo(options =>
{
    // ...
    options.OnMessage = msg => msg.WithSourceCodeFromPdb();
});

Now, when logging the same error, we get the Source code tab shown in the elmah.io UI:

For the final step, I'll generate a new suggestion from ChatGPT:

As you can see, the result varies from the previous by now including potential fixes based on your own source code. Looking at suggestions based on your existing code typically beats most generic blog posts out there.

To finish this post, I want to put a few words on using ChatGPT for your code. As I already mentioned, installing this app will share details about your errors and maybe even source code and SQL. You will need to decide if this is something you want to do. The elmah.io Enterprise supports locally hosted models, which means that your data won't leave elmah.io's data center.

Also, blindly trusting suggestions and generated code from ChatGPT can be exactly as dangerous as trusting recommendations in random blog posts or Stack Overflow responses. As developers, we should always be critical of code fixes like this. With that said, we've found the ChatGPT app to be extremely helpful for our own logs already. It has become a normal part of our process for fixing errors on elmah.io.

See how we can help you monitor your website for crashes Monitor your website