Export data from Google Analytics with .NET 💾

We are using a couple of dashboards to monitor various metrics on elmah.io. One of these metrics being data from Google Analytics. To continuously pull data from Analytics and storing them in a local store for easy access, we have implemented an export job in C#. In this post, I'll introduce you to how we did that.

To export data from Google Analytics, I will recommend you to look at the Google.Apis.AnalyticsReporting.V4 NuGet package, Google's official package for interacting with Google Analytics from .NET. The code is currently in maintenance mode, but that doesn't mean that it's obsolete or anything like that. Google still fixes critical bugs and releases minor versions monthly.

You will need to apply credentials to access your analytics data, why I'll start by showing you how to set up a new set of credentials.

Go to Google Developer Console and create a new API project. If you don't have a Google account already, you will be prompted to create one.

When the project is created, click the button named ENABLE APIS AND SERVICES. You should be able to find an API named Google Analytics Reporting API.

Click ENABLE and the reporting API is added to your API project. Select the Credentials menu item and click the CREATE CREDENTIALS button. For this example, we'll create a server-to-server integration which will require you to select the Service account type. Input an account name and ID and click the CREATE button:

Skip the Grant this service account access to project and Grant users access to this service account sections by clicking the CONTINUE and DONE buttons.

Click the newly created service account and either add an existing SSH key from your machine or generate a new one. We'll need the key in a minute when we start coding. For this example, I'll create a new key by clicking ADD KEY and Create new key. Select P12 in Key type and click Create:

This will download a certificate file that we can bundle with our C# code.

The only missing thing before getting to write some code is to allow the new service user access to our Google Analytics data. To do so, copy the service account email from Google Developer Console and head over to Google Analytics to launch the Admin tool. Select the account you want to allow access to and click Account User Management. There, you can add the service account user with the Read & Analyze permission:

Ready to write some code! I'll create a .NET 5 console application to demonstrate but you can integrate Google Analytics in whatever .NET project type you'd like. To illustrate how to export data, I want to pull the number of unique users visiting a website yesterday.

Start by creating a new project and install the Google.Apis.AnalyticsReporting.v4 NuGet package. Also, add the downloaded certificate in your project root. If you right-click the certificate file in Visual Studio and select Properties you can instruct .NET to copy the file to the output directory by selecting Copy if newer:

To use the classes provided by Googe, we need to include the following namespaces:

using Google.Apis.AnalyticsReporting.v4;
using Google.Apis.AnalyticsReporting.v4.Data;
using Google.Apis.Auth.OAuth2;
using Google.Apis.Services;
using System;
using System.Collections.Generic;
using System.Security.Cryptography.X509Certificates;

To call the Analytics API you will need to create what Google calls an Initializer. The initializer includes the ID of the project you created on Google Developer Console as well as the scopes needed by your application:

var initializer =
    new ServiceAccountCredential.Initializer(
        "dotnettest@dotnet-test-301611.iam.gserviceaccount.com");
initializer.ProjectId = "dotnet-test-301611";
initializer.Scopes = new string[]
{
    AnalyticsReportingService.ScopeConstants.AnalyticsReadonly
};

The email address must match that of the service credential that we created on Google Developer Console and authorized on Google Analytics.

Next, we need to load the certificate to authenticate the user:

var priv =
    initializer.FromCertificate(
        new X509Certificate2("dotnet-test-301611-ce23d8dc5916.p12", "notasecret"));
var creds = new ServiceAccountCredential(priv);

The certificate is created from the .p12 file and a password of notasecret is specified. This is the default password provided by Google. You probably want to change the password to something else in a real application.

The credentials can be provided the AnalyticsReportingService that we will use in a bit:

var service = new AnalyticsReportingService(new BaseClientService.Initializer
{
    HttpClientInitializer = creds,
});

We have finally arrived at the point where we can start pulling information from Google Analytics. Pulling a single metric like users requires more lines than you would think. Understanding this part is probably the hardest part when interacting with Google Analytics. I'll paste all of the lines to make the request and go over them line by line:

var dateString = DateTime.Today.AddDays(-1).ToString("yyyy-MM-dd");
DateRange dateRange = new DateRange()
{
    StartDate = dateString,
    EndDate = dateString
};
Metric users = new Metric
{
    Expression = "ga:users"
};
ReportRequest reportRequest = new ReportRequest
{
    ViewId = "VIEW_ID",
    DateRanges = new List<DateRange>() { dateRange },
    Metrics = new List<Metric>() { users },
};
List<ReportRequest> requests = new List<ReportRequest>
{
    reportRequest
};
GetReportsRequest getReport = new GetReportsRequest
{
    ReportRequests = requests
};

GetReportsResponse response = service.Reports.BatchGet(getReport).Execute();

var result = response.Reports[0].Data.Totals[0].Values[0];
Console.WriteLine($"Users: {result}");

In the first few lines, I define the date range to search. I'm including yesterday in both start and end. Next, I define which metric to fetch. This is done on the format ga:metric where metric is the name of the metric to fetch. In this case the count of users. The date range and metric are combined in a new ReportRequest alongside the ID of the view to fetch data from. The View ID can be found by click View settings in the Google Analytics admin. Finally, I wrap the ReportRequest in a list and again in a GetReportsRequest object. This endless wrapping seems a bit over the top, but I guess that maps to how the API is structured. The GetReportsRequest object is provided to the BatchGet method and the Execute method is called to make the API request. In the last few lines, I unwrap the users metric which is hidden deep within an object structure. I guess this structure makes more sense when fetching more data, compared to our single metric here.

When looking over the code, a lot of lines are needed to fetch a single metric. I would probably have designed something simpler, but I also acknowledge that there are more advanced use cases out there, that will benefit from the deep object structures in play here.

Here's the full sample:

var initializer =
    new ServiceAccountCredential.Initializer(
        "dotnettest@dotnet-test-301611.iam.gserviceaccount.com");
initializer.ProjectId = "dotnet-test-301611";
initializer.Scopes = new string[]
{
    AnalyticsReportingService.ScopeConstants.AnalyticsReadonly
};
var priv =
    initializer.FromCertificate(
        new X509Certificate2("dotnet-test-301611-ce23d8dc5916.p12", "notasecret"));
var creds = new ServiceAccountCredential(priv);

var service = new AnalyticsReportingService(new BaseClientService.Initializer
{
    HttpClientInitializer = creds,
});

var dateString = DateTime.Today.AddDays(-1).ToString("yyyy-MM-dd");
DateRange dateRange = new DateRange()
{
    StartDate = dateString,
    EndDate = dateString
};
Metric users = new Metric
{
    Expression = "ga:users"
};
ReportRequest reportRequest = new ReportRequest
{
    ViewId = "VIEW_ID",
    DateRanges = new List<DateRange>() { dateRange },
    Metrics = new List<Metric>() { users },
};
List<ReportRequest> requests = new List<ReportRequest>
{
    reportRequest
};
GetReportsRequest getReport = new GetReportsRequest
{
    ReportRequests = requests
};

GetReportsResponse response = service.Reports.BatchGet(getReport).Execute();

var result = response.Reports[0].Data.Totals[0].Values[0];
Console.WriteLine($"Users: {result}");

elmah.io: Error logging and Uptime Monitoring for your web apps

This blog post is brought to you by elmah.io. elmah.io is error logging, uptime monitoring, deployment tracking, and service heartbeats for your .NET and JavaScript applications. Stop relying on your users to notify you when something is wrong or dig through hundreds of megabytes of log files spread across servers. With elmah.io, we store all of your log messages, notify you through popular channels like email, Slack, and Microsoft Teams, and help you fix errors fast.

See how we can help you monitor your website for crashes Monitor your website