AI and ML can help utilities understand unstructured data and improve grid reliability

Recently, an executive at a major electric utility confided in me a professional observation he had made of his organization’s data. 

“It’s a mess.”

He went on to clarify that his organization’s data wasn’t a complete mess. Some of it, he explained, was rather well organized and quite useful. 

But a lot of it, he confessed, was…

This is where I stepped in with a more encouraging assessment.

The word I used was unstructured, which I explained was not only a common type of data but also one that (with the right tools) can be organized, analyzed, and put to use to help utilities make assessments and decisions to improve grid reliability. 

While a utility’s data can take many forms, it typically falls into two general categories: “structured” and “unstructured.”

Structured vs Unstructured Data

Structured data is typically numerical or categorical and is organized in a way that makes it easy for computers to read, interpret, and analyze. 

Utilities may have data stored in a work management system that contains clearly defined pieces of information such as work order numbers, descriptions, and status–all of which can be considered structured data.

Unstructured data, on the other hand, is not organized in a neat and tabular format and requires additional work to be understood and analyzed. 

A utility’s unstructured data might include documents, images, or even more complex sets such as Light Detection and Ranging (LiDAR) that are not organized in an easily-readable format and therefore requires some effort before the data can be put to use. 

Benefits of Understanding Unstructured Data for Electric Utilities

Although it can be difficult to interpret initially, unstructured data is often more valuable than structured data as it can provide additional, unanticipated insights. 

Consider imagery data utilities acquire in the field to assess their grid’s physical infrastructure. There are many forms incoming field data may take including digital photography taken by fielders on the ground, aerial photography taken by remote control drones, as well as LiDAR. 

Here we have a classic set of unstructured data whose value has enormous potential. The problem is that, as unstructured data, it is unorganized and can not be put to any kind of immediate use by the utility. 

Unless the utility is willing to take on an enormous amount of human labor to identify key aspects of the imagery, the data will be of little help to improve grid reliability. 

Using Structured Data to Create a Predictive Analytics Model

The electric utility’s ultimate goal is to increase the reliability of the bulk electric system. To go about tackling a small piece of this large goal, a utility might consider creating a predictive analytics model to prioritize asset replacement of poles.

This model will make use of both structured and unstructured data. 

The utility’s current asset management system will likely have key structured data such as a pole install date, pole type, and other fields. With some simple feature engineering, it will be possible to determine which fields contribute most to pole health and then create a proactive maintenance plan based on this structured data.  

Perhaps there are additional open-sourced datasets that can be leveraged to improve performance. By adding weather or geographical datasets, additional features can be added to the model,  creating a more complete picture of what most affects poles. 

Now let’s look at how unlocking the value of unstructured data can improve our model.

Using A.I. and Machine Learning Technology to Unlock Unstructured Data’s Value

Consider images of poles a utility may have from a past joint-use audit. As is, they are merely untagged images with no further insights or meta information. This is unstructured data to the letter of the definition. 

It would take considerable work for an individual (or many individuals, for that matter ) to go through the images manually and identify key assets—i.e. what equipment is attached to the poles—for analysis. 

Here is where technology comes in, adds structure to unstructured data, and gives the utility insights at scale.

A data-agnostic AI tool such as IKE Insight, can make easy work of identifying what pieces of equipment are on your poles. That’s because such a tool can work with any type of image the utility may have collected in the field such as drone, thermal, LiDAR, 2D, 3D, video, or another form of aerial photography. 

The utility can then apply rules or attributes to the data to give structure to the intelligence that the tool is going to create from the data. Then, the tool uses its artificial intelligence features and machine learning capabilities to create actionable insights from the bulk data. 

In the case of IKE Insight, the AI involved identifies the objects that are on the poles by using training data, which is a small amount of annotated data used to predict what objects are in new images.

Such insights are dependent on the attributes the utility defined at the outset of the process, but now the unstructured data has become structured and can be used as the utility sees fit. 

The applications for structured data created by IKE Insight are almost infinite, but a few of the more common use cases include:

• Utility pole audits-IKE Insight allows for a higher volume and efficiency of audits for joint use and pole transfer agreements, resulting in improved cash flow and regulatory compliance.
• Pole Inspections and asset planning–Networks can gain even greater visibility to detect line degradation and vulnerabilities using object and anomaly detection, as well as plan network expansion/upgrades. 
• Unlocked value from idle data–Existing imagery and other unstructured data from previous data collection projects can yield new benefits.

Additionally, the utility can add business logic (for example,  the more objects the pole contains, the more weight it should be assigned in the predictive model)  that can output a prioritized list of circuits for upgrades or maintenance.

Using this method, the manual effort of going out and collecting this data (again) is eliminated, ensuring that limited human resources are working on the most important tasks.

When it comes to unlocking the value of unstructured data, a utility can never have too many resources in its corner, especially given the amount of work that must be done to modernize the electric grid. 

AI and machine learning technology that allows more work to be done with fewer human resources behind it is exactly the kind of boon the industry needs right now to turn idle, unstructured data into actionable insights that can help utilities create a more reliable grid for today and tomorrow.  

Matt Fernandez is a Project Manager and Certified Scrum Master for ikeGPS. He has eight years of experience with a large public utility, performing various roles from managing IT projects to capital planning.  He holds a bachelor’s degree in Information Systems Management from Duquesne University and an MBA from The University of Akron.