How Monitoring and Diagnostics Centers Benefit the Power Industry
Credit to Author: Aaron Larson| Date: Thu, 01 Dec 2022 05:11:00 +0000
GE has been operating a monitoring and diagnostics center for the power generation sector for more than 25 years. From modest beginnings, the facility now monitors about 1,000 power plants around the world, providing a number of significant benefits for customers.
To get a better understanding of the day-to-day activities at GE’s power generation monitoring and diagnostics (M&D) center, POWER interviewed Justin Eggart, chief engineer of Services Technology with GE Gas Power, and Rahul Chadha, vice president of Technology with GE Digital. Eggart has more than 30 years of experience with GE, including running the M&D center for a number of years. Chadha originally joined GE to work at the M&D center 23 years ago. During his time there, he helped move the center from the company’s headquarters in Schenectady, New York, to Atlanta, Georgia, where it is today (Figure 1). Now, with GE Digital, Chadha considers himself a “service provider” to the M&D center—his group runs the APM (Asset Performance Management) application that the center uses to monitor plants around the globe.
POWER: How has GE’s M&D center evolved over time?
Eggart: We’ve been operating the power M&D center now for over 25 years, and certainly, there have been a lot of changes in that time. When we first started in the mid-1990s, we set up to focus on identifying some problems we were having with a handful of gas turbines. Today, we monitor about 1,000 power plants around the world, which includes 2,500 gas turbines, another 500-plus steam turbines in combined cycle, and a lot of balance of plant equipment in all of those plants. So, the scale has changed tremendously.
When we first started monitoring—from a tools perspective—we were essentially manually monitoring. We were streaming data and looking at it on a screen. Today, we’re highly automated. We use advanced digital tools and analytics to automate much of our anomaly detection work, our algorithms, if you will, that help us find issues that our subject matter experts can then focus on. We’re running 300-plus analytics in real time within the digital APM software. We’ve got about 600 terabytes of data stored that we use to help us troubleshoot issues.
Lastly, the technology that’s available to us has also changed tremendously—both the technology of the equipment we’re monitoring, as well as the technology we use to monitor it. When we first started, we had five people. Now, we have a team of 100 people. So, the team has scaled up significantly. One thing that has helped us manage that, though, is all of the automation and tools that we have available to us now to improve productivity.
POWER: How does the monitoring process work? Does the anomaly detection system send engineers notifications that a parameter is out of range and then they investigate to identify a cause?
Eggart: Yeah, that’s right. We use the automation, the analytics and the software, if you will, to help us identify the issues—when a parameter might be out of bounds or a process might be out of bounds. That then flags our subject matter experts [SMEs] to go take a closer look and make a recommendation to the customer as to what to do.
A customer can still call us directly and ask us to look at data—and we do that all the time—but essentially, you’re right. That’s how the process works. The automation identifies potential issues and a subject matter expert takes a quick look at that, figures out what the issue might be, and responds to the customer with advice and recommendations on what to do about it.
Chadha: I’d like to take you through a little bit of a technology conversation here as well. To start with, we stream 30 billion points from 86 countries across the globe—that’s our footprint. We are getting all these data points. We are looking at high-low limits—there are critical limits that we watch for—but from day one we have employed significant data science models called digital twins, where we are looking for a normal condition of a machine with respect to its settings. And, if there’s a deviation from normal bands, we have married that with physics-based models on where the deviation is critical enough for our team of experts to go monitor the alarms. So, it’s not just the data coming out of our sensors. We have built a lot of artificial intelligence tools recently, but more complex physics tools over the years, to monitor the specific failure modes on our machines.
POWER: Do most of your customers realize they have a problem or are you identifying problems that they aren’t even aware of?
Eggart: I would say that more than half of the time we are identifying problems they don’t even know they have yet. So, for example, we might see degradation in a valve, or we see contamination in a fuel line, or things like that. The customer doesn’t even realize operationally that they have a problem. Certainly, there are situations where customers come to us and ask for help, and we gladly help them, but I would say more than half of the things we identify are things they don’t even know about yet.
POWER: What is a typical day like for a staff member at the M&D Center? Are they assigned to monitor specific plants or is there a queue that they pick from to work on the next available problem?
Eggart: In our main M&D Center here in Atlanta, we run three shifts. It’s a 24/7/365 center, so a typical technician will work an eight-hour shift. And the way we have it set up is they’re not dedicated to specific plants. We continuously run these analytics against the data as it streams in, and that identifies issues and puts them in a queue to be worked. The technicians pick the next thing off the queue, and go and work on it.
The way that works is if the technician can identify and solve the problem immediately, they will do that with the customer. If it’s a more complicated issue that they can’t solve right away, they’ll kick it to one of our subject matter experts that are nearby and they’ll work on the issue while the technician moves to the next issue in the queue.
POWER: So, it sounds like there are steps, upon which issues can be escalated. Is that accurate?
Chadha: I would put it another way, which is, our first line of defense is to ensure that the data is accurate—there is no sensor drift or data quality problem. Once we establish that an issue is real, typically what happens is the technicians kick it to tier-two engineers, who are subject matter experts in specific technologies, such as combustion technology, compressor technology, or balance-of-plant equipment. These engineers are the ones that take the root causes and recommendations back to the customers. Technicians are experienced young engineers, who have worked in the power industry.
And one thing to add: the SMEs are global. We have a good team here in Atlanta, with our services engineering team, but we also have engineers in 13 countries, who are associated with the M&D Center (Figure 2). These engineers are part of our tier-two support, looking at specific fleets within the regions, but also globally as well.
POWER: While your main M&D center is in Atlanta, how many other centers do you have around the world?
Eggart: Realistically, from an M&D perspective, we have about seven centers or so around the world. The way I would think about it is: GE being a large global company, we have subject matter expertise in multiple locations around the world. Our main center is here in Atlanta—call it our tier-one operation center—so everything comes into here first.
And, then, we may open up a case in our case management system, and we can assign that case to somebody anywhere in the world. If it’s the middle of the night in Atlanta, we might assign that case to a subject matter expert in India, for example. We can do that in real time. Our team around the world has access to the dataset and the tools that we use, so they can work those issues from anywhere, but they’re concentrated in our large centers around the world.
Chadha: Additionally, we have very big customers, who also have centers using our technology—they use our cloud APM—and they collaborate with our central M&D center on a regular basis.
POWER: What are some of the more common problems you find? Do you regularly help customers improve their heat rate or identify bearing failures or what are the typical issues you deal with?
Eggart: The failures vary widely. One of the most common things might be a sensor failure. For example, a sensor may fail in the fleet and be giving some erratic data or causing some control problem. That’s a common issue. Valve failure is another very common issue where we see valve degradation—the valve is not closing or opening as quickly as we would expect.
Things like a bearing failure are much less common. A lot of the most common issues tend to be around the accessories to the power turbine, so sensors, valves, tubing or piping failures, fuel issues, fuel contamination, things like that. Heat rate? We do fairly often make recommendations about how to improve heat rate. Often, that’s driven by degradation in the gas turbine, from dirty air, for example, which makes the compressor dirty and makes it degrade. Things like that.
Chadha: We also provide unplanned shutdown support to our customers. So, anytime one of our units goes offline, we have a rich dataset available to us—milli-second data—that we are analyzing as the customer is taking care of its turbine. And we provide an immediate recommendation.
Our goal is to get back to the customer within 20 minutes. We allow five minutes for data collection and 15 minutes for them to do their work so that they are not distracted by calls from GE, but our goal is to get back with a recommendation to ensure that they understand the root cause and if it is safe to restart the machine.
Beyond that are the equipment failures, as Justin described, but also, I would say, a large portion of our focus is on heat rate. Heat rate comes in two ways. One is where we have anomaly detection algorithms that are running in real time. So, we run what we call a thermodynamics model of our gas turbine in real time every five minutes. And if there’s a major degradation, we get an alert in the system and we follow the process.
In addition to that, we have monthly reports. Heat rate typically degrades over time. Unless the degradation is associated with a reliability failure, it’s usually over time that you will find heat rate fuel problems. And those are the findings that come as a monthly report identifying where a plant is operating with respect to standards and how they have degraded or improved in their operations. With that, we provide insights and recommendations.
POWER: I assume most of your customers operate GE gas turbines. Do you also monitor non-GE equipment?
Eggart: We do. The overwhelming majority of our fleet are GE gas turbines. However, we do have customers who have a combination of technologies, so we are on occasion supporting a Siemens gas turbine, for example, or an older Westinghouse gas turbine. We do have a few of those installations, but the majority are GE.
POWER: What do you see as the biggest benefit of having an M&D center?
Eggart: One nice thing about having a remote monitoring center is that you can concentrate the subject matter expertise in a particular area. For example, if I have a gas turbine combustion expert and that person is located at a site, such as a two-gas-turbine combined cycle plant, that combustion expert is focusing on two gas turbines. Whereas, in a remote monitoring center, that expert can lend their expertise to hundreds of gas turbines.
And that also then allows them to learn much more quickly. So again, at a site, they may see a particular failure once in a lifetime. But in a center like this, they may see that failure multiple times, which allows them to learn much more quickly, develop their expertise, and improve their recommendations. So, I think there’s a lot of value in some of these remote centers where you can combine that subject matter expertise and scale it across a fleet like we’ve done.
Chadha: With COVID-19, when people were not able to get to sites or get to their work locations, I think it was very telling and very important for our customers to have this center available as the eyes and ears to their datasets and their data plans for more reliable operations. And a lot of our customers are now very interested in having their own centers collaborating with this center.
The game has changed over the last few years, and GE has been doing this for 25 years now and in about 86 countries. It just doesn’t come by itself. Every country has their own cyber law, data law, GDPR [General Data Protection Regulation] laws. And this team has done significant work over the years, with investment from GE, to ensure that we are meeting each and every standard. So, the scale of it is what matters, as well as the outcomes that our customers are looking for. The expertise that this brings is something that our customers are trying to build as well, helping their own end-users or end-customers with that stuff.
The second thing I will draw around concerns IIoT [Industrial Internet of Things] centers. I think this is one of the big use cases GE had when we started our digital journey. It has been a showcase, not just for GE Gas Power, but also for several other businesses. So, the scale of it, it didn’t just come overnight, it’s like a scale that GE built over time. And this is the kind of scale that people are looking for, so that they can bring all that data together, and then look for the insights from those data that the next-gen big-data technologies can provide so they can then transfer that back to the end-users.
I think GE Power’s M&D center has been a pioneer in that. They have not been static. What was a game-changer for us was a Windows 95 machine with a little modem that could stream data to a data center within America. From that Windows 95 machine, we shifted to Oracle, to historian-type technologies, and now to big-data AWS [Amazon Web Services] technology. That has been our evolution over the last 25 years. Every five years we have changed our technology just to keep up with the demand.
—Aaron Larson is POWER’s executive editor.
The post How Monitoring and Diagnostics Centers Benefit the Power Industry appeared first on POWER Magazine.