How to write to a file with C# - StackOverflow doesn't get it right

Today's tip isn't exactly about a hot new version of ASP.NET Core or anything like that. I often find myself googling "simple" questions like how to write text to a file in the quickest and effective way. Most results are either blog posts using .NET 2 as an example, or StackOverflow answers from 2010. Common for all samples is that only the essential lines of code are added, and no focus on crucial aspects like error handling is highlighted. This post is my attempt to sum up what I've learned over the years.

File.WriteAllText

The first example of writing a string to a file is using the File.WriteAllText method. The reason I mention this as the first example, is because it's what 95% (maybe more?) of people are doing. And I understand why since I have been doing this for years. Take a look at the simplicity of this approach:

var writeMe = "File content";
File.WriteAllText("output.txt", writeMe);

The example produces a file named output.txt with the content of the writeMe variable. This approach can be fine for smaller text files, but shouldn't be the default choice when handling larger amounts of data.

Let's create a scenario for the rest of the post, where a large amount of data should be stored and use WriteAllText to write the data to a file:

var sb = new StringBuilder();
var lines = Enumerable
    .Range(1, 1000000)
    .Select(i => $"Line number {i}")
    .ToList();
lines.ForEach(i => sb.AppendLine($"Line number {i}"));

var content = sb.ToString();
File.WriteAllText("output.txt", content);

In the example, I generate 1 million lines and store them in the output.txt file. On my machine, the code takes around 100 ms. That's pretty impressive for most scenarios, but I'm sure we can do better.

File.CreateText

Where the WriteAllText method writes everything at once, you may need to write in batches. Let's reuse the StringBuilder from before to illustrate how this can be done using the File.CreateText method:

using (var writer = File.CreateText("output.txt"))
{
    foreach (var line in lines)
    {
        writer.WriteLine(line);
    }
}

The CreateText method returns a StreamWriter object that we can use to publish multiple lines to a file or even write to the file one char at a time. When disposing of the StreamWriter, the buffer is flushed to the underlying stream, and the filehandle is released. You can manually call the Flush method to get the same result.

Appending text

As an alternative to the previous methods or if you simply need to append text to an existing file, there are append-methods available as static methods too:

File.AppendAllText("output.txt", content);
File.AppendAllLines("output.txt", lines);

Both methods append text either as a single string or multiple lines in a single write. You can split lines in multiple iterations using the AppendText method:

using (var writer = File.AppendText("output.txt"))
{
    foreach (var line in lines)
    {
        writer.WriteLine(line);
    }
}

Be aware to only call AppendText once during the duration of having to write content to a file. I sometimes see code like this:

foreach (var line in lines)
{
    using (var writer = File.AppendText("output.txt"))
    {
        writer.WriteLine(line);
    }
}

As we will see later in this post, this will be very performance heavy since the file is touched multiple times on the disk.

FileStream

I've seen it so many times. Customer complaints that the UI "hangs" when clicking something the involves reading from or writing to a large file. You probably know where this is going, right? Async!

A good choice for writing asynchronously to a file from .NET is using the FileStream class. Let's port the example from before to use FileStream. First, a synchronous example:

using (var stream = new FileStream(
    "output.txt", FileMode.Create, FileAccess.Write, FileShare.Write, 4096))
{
    var bytes = Encoding.UTF8.GetBytes(content);
    stream.Write(bytes, 0, bytes.Length);
}

In the example, I wrap the FileStream in a using, which will (not surprisingly) dispose of the stream once done with it. You mainly use the Write method, which writes one or more bytes to the underlying stream. You can call the Write method multiple times, which makes this a good choice when needing to iterate over something like database records and write each record to the same file.

Using FileStream is typically faster than the methods from the last two examples. When storing the data generated for this post, using the FileStream is around 20% faster than the other examples. And even more quickly when running on .NET Core.

Now for the async version:

using (var stream = new FileStream(
    "output.txt", FileMode.Create, FileAccess.Write, FileShare.Write, 4096, useAsync:true))
{
    var bytes = Encoding.UTF8.GetBytes(content);
    await stream.WriteAsync(bytes, 0, bytes.Length);
}

Notice how I have added true (useAsync parameter) as the last parameter to the FileStream constructor. This tells the FileStream to use asynchronous IO. I then call the WriteAsync method instead of the Write method from the previous sample. Also, remember to await the method.

When looking at the performance of the async code in the last example, performance corresponds to the two first examples. So why use the async FileStream when slower than the synchronous one? To avoid blocking the main thread, of course.

Performance

Using the right method for the job is very important, rather than just throwing yourself on the first answer from StackOverflow. To showcase the difference in performance on the methods listed in this post, I'll use appending to a file since that is the scenario I often see people getting wrong. I already showed the code but let's recap with a requirement to append 10,000 lines to a file:

var lines = Enumerable
    .Range(1, 10000)
    .Select(i => $"Line number {i}")
    .ToList();

// Call File.AppendAllText 10,000 times
{
    var stopwatch = new Stopwatch();
    if (File.Exists("output.txt")) File.Delete("output.txt");

    stopwatch.Start();
    foreach (var line in lines)
    {
        File.AppendAllText("output.txt", line);
    }
    stopwatch.Stop();

    Console.WriteLine(stopwatch.ElapsedMilliseconds);
}

// Call File.AppendAllLines 1 time
{
    var stopwatch = new Stopwatch();
    if (File.Exists("output.txt")) File.Delete("output.txt");

    stopwatch.Start();
    File.AppendAllLines("output.txt", lines);
    stopwatch.Stop();

    Console.WriteLine(stopwatch.ElapsedMilliseconds);
}

// Call File.AppendText 10,000 times
{
    var stopwatch = new Stopwatch();
    if (File.Exists("output.txt")) File.Delete("output.txt");

    stopwatch.Start();
    foreach (var line in lines)
    {
        using StreamWriter streamwriter = File.AppendText("output.txt");
        streamwriter.WriteLine(line);
    }
    stopwatch.Stop();

    Console.WriteLine(stopwatch.ElapsedMilliseconds);
}

// Call File.AppendText 1 time
{
    var stopwatch = new Stopwatch();
    if (File.Exists("output.txt")) File.Delete("output.txt");
    stopwatch.Start();
    using (StreamWriter streamwriter = File.AppendText("output.txt"))
    {
        foreach (var line in lines)
        {
            streamwriter.WriteLine(line);
        }
    }
    stopwatch.Stop();

    Console.WriteLine(stopwatch.ElapsedMilliseconds);
}

As you can see from the code, I have made 4 different attempts to write 10,000 lines to a file. The result looks like this on my machine:

Not surprisingly, touching the files multiple times by calling File.AppendAllText multiple times or embedding the StreamWritter within the foreach instead of the other way around comes with a huge cost in terms of performance.

.NET Core/.NET 5

We briefly touched upon .NET Core/.NET 5 in one of the previous examples. I'm happy to tell you that the APIs we have used in the post all share the same signature in .NET Core/.NET 5. This means that you can switch your project to .NET Core/.NET 5 and recompile all of the examples from above without any compile errors.

When running the examples, all of the code runs around 5-10 ms faster. The difference between .NET Full Framework and the new .NET will, of course, be dependent on your hardware.

Exception handling

An often overlooked (yet so important) aspect of writing to a file is exception handling. A lot of things can go wrong when interacting with the file-system. There are two groups of exceptions you need to worry about:

1. Problems with the path.
2. Problems while writing to disk.

Before we dig into each group, let's add exception handling to the File.WriteAllText example:

try
{
    File.WriteAllText("c:\\temp\\output.txt", "Hello World");
}
catch (DirectoryNotFoundException dirNotFoundException)
{
    // Create and try again
}
catch (UnauthorizedAccessException unauthorizedAccessException)
{
    // Show a message to the user
}
catch (IOException ioException)
{
    logger.Error(ioException, "Error during file write");
    // Show a message to the user
}
catch (Exception exception)
{
    logger.Fatal(exception, "We need to handle this better");
    // Show general message to the user
}

Please note that implementing control flow using exceptions is considered a bad practice. While the code above serves well as an example, you probably want to check common scenarios like a directory not found before trying to write to a file.

Problems with the path

There is a range of different issues that can cause exceptions to happen when dealing with files. I have already blogged about a couple of these here: Debugging System.IO.FileNotFoundException - Cause and fix and Debugging System.UnauthorizedAccessException (often followed by: Access to the path is denied).

You should always wrap code writing to a file in a try-catch and decide what should happen if the following exceptions occur:

• DirectoryNotFoundException
• UnauthorizedAccessException

Some exceptions can be implemented as auto-healing by your code (like creating a missing directory and writing to the file again), while others will result in a message to the user or a log message to be written to your log.

Problems while writing to disk

Other exceptions deal with problems while writing content to the file. Once you know that the file can be created and that the directory to contain the new file exists and that the process has sufficient access, a large number of things can still go wrong.

Most problems happening during the actual write, are thrown as an IOException. IOException is the base class for most file-related exceptions anyway, so make sure to catch each sub-class to be able to handle different scenarios accordingly.