A LogicMonitor Product

The 2022 Airbrake Error Data Report

In the last year and a half, we've seen the unprecedented adoption of digital products. IDC estimates that by 2023, developers will have created over 500 million new cloud-native products. That's a lot of code. And with this amount of code, there will be countless errors, no matter how good of a QA system you might have.

That's where The 2022 Airbrake Error Data Report comes in. The data set analyzed in this report from 2021 spans almost 2 billion error occurrences and 44k unique error types from Airbrake projects.

 

Errors Divided by Application Language

Error Report Underlying Data Charts-3

What Did We Find?

 

Most software projects run with a manageable number of errors

According to our data, most projects run with a manageable number of errors. 

A couple of interesting notes:

  • Around 90% of all projects have less than 10 different types of errors per month
  • Around 5% of all projects produce multiple thousands of errors every month

In addition to a manageable number of errors, these projects are able to keep adding features to their code without growing their error counts drastically.

Number of Unique Errors Per Project

Number of unique errors per project-2

One-third of all software projects fix their errors in <2 weeks

The fact that one-third of all software projects fix their errors within the same sprint mirrors how teams prioritize errors based on their criticality. 

The pie chart on the right demonstrates this. For example:

  • 16.8% of all projects closed errors with relative immediacy (within 24 hours)
  • 32% of all projects manage to close errors within the same sprint (2-week window)
  • 37.5% of all projects closed errors after 90 days

Some issues require immediate attention, while others can be factored into sprint plans or be left alone.

*We determined the length of error fixes from the first occurrence of the error and the last occurrence of that error, which signaled the time it took for a team to resolve that error.

Number of Days to Fix Errors*

Error Report Underlying Data Charts 2

More errors are captured by backend languages

Medium Number of Errors Per Project and Language (12 months)

Error Report Underlying Data Charts-18

According to our data, languages typically associated with backend projects seem to capture more errors than frontend languages. What might be causing this?

    • Are frontend languages less mature when it comes to error logging? Perhaps they don't capture as many errors compared to backend languages? 

 

    • Can the high number of errors in backend languages be explained by the complexity and interdependence of microservices? Simply put, do these errors make more noise? 

 

    • Could the number of errors reflect the learning process as developers practice writing effective code? For example, Golang is a relatively new and popular language. Perhaps the errors we're seeing are the result of try-fail pattern practice?

 

What do you think? Tweet your opinion and tag @airbrake

Common Errors and Takeaways by Language

 In this section, we'll dive into patterns and errors associated with the following programming languages:

Golang, Ruby, Java, JavaScript, and Python.  

 

Golang

Golang projects tend to have errors that might not be errors

In Golang, expect some transient errors that will resolve themselves. This is due to issues related to design patterns within a micro-service environment. For example, 57% of errors involve the MPD client failing to connect, most likely due to a migration event. Because of this, knowing WHEN to raise an error in Go is key.

Error Report Underlying Data Charts-12

Ruby

Double-check Ruby projects that they have the right objects and arguments

The top errors in Ruby projects all tend to have the same theme: a missing or unexpected object or argument. The best way to fix this is to double-check operations before an operation runs. The idea is to make sure the job can be done before attempting to do the work. Save time and use an error monitor to check for missing or incorrect objects. 

Error Report Underlying Data Charts-13

Java

Java programmers face two challenges: objects and network errors

As an object-oriented language, Java tends to have issues declaring objects and passing data. For example, the majority of Java projects include the "NullPointerException." This exception tends to pop up when the object does not point anywhere. The amount of "IOExeption" errors within Java projects also show issues with network connections. If you are a Java programmer, focus on declaring objects and resource management of external dependencies. 

Error Report Underlying Data Charts-15

JavaScript

External dependencies are challenging in JavaScript projects

What do the  "NotFoundError," "SecurityError," and "TimeoutError" have in common? They're all related to failures interacting with external dependencies. Keep this in mind as you're coding in JavaScript. "Is my code written correctly so that it can read cookies?" "Do I have permission to read these cookies?" "If there's an increase in transactions being timed out, could there be an issue with the UI?"

Error Report Underlying Data Charts-16

Python

Python programmers should focus on data processing bugs

In Python projects, errors related to data processing are prevalent. Many projects are issuing errors like "IndexError," "JSONDecodeError" and "DataError," which suggests that data processing is a common and challenging task for Python. The simple solution here is to look at your data processing setup. Is the problem the code or something on the end? 

Error Report Underlying Data Charts-17

Conclusion

 

Errors are a part of the daily life of software engineering teams and projects. Whether you're a frontend, backend, full-stack developer, errors are inevitable, but you can control how you respond to them. As we saw in this report, many projects do fantastically well in responding to errors in a swift manner, likely leading to good user satisfaction, low cost, and limited technical debt associated with latent errors, while others have a good opportunity to shape up their debugging process. 

About Airbrake and This Data

Airbrake was founded in 2008 (originally called Hoptoad) and defined the error monitoring category. Since then, Airbrake has developed into a full suite of error and performance monitoring capabilities - but kept true to their saying "by developers for developers." In 2021, Airbrake was acquired by LogicMonitor, and is now part of a larger family of monitoring solutions, but operates independently.

As an error-monitoring platform, our goal is to give our customers the tools they need to succeed. In the case of this error report, we saw consistent patterns associated with particular languages. Using the almost 2 billion errors Airbrake received, we crafted a report that we hope helped you understand your own code and application a little bit better. 

Have feedback about this report? A suggestion? Comment? Reach out to us at customersuccess@airbrake.io!

 

A LogicMonitor product.