Connecting Cloud-based Quality Control to the Edge

Summary

Machine vision systems can recognize superficial aberrations in manufactured goods—like lines, spots, holes, color discrepancies and more—that would typically require time-consuming manual inspection. However, these systems require specialized programming and maintenance, which can make them difficult to implement.

California-based Intrinsics Imaging solves this problem with its Heijunka Vision analytics-as-a-service software. Instead of programming specialized cameras, Heijunka provides a library of image processing and machine learning algorithms running in the cloud that allow any IP camera to perform intelligent defect detection. Intrinsics works with customers to tailor analytics to their specific objectives, then provides a concierge level of service so that customers no longer need to train, adapt, or maintain their vision system.

Typically, Heijunka integrates with SCADA systems to create analytics dashboards, alarms, and quality control actions. But when Intrinsics was approached by a customer hoping to integrate Heijunka directly into process controls, it looked to Opto 22’s groov RIO edge I/O for a way to connect the cloud to the edge.

Securing a path to the edge

For applications like manufacturing asphalt roofing and other kinds of building products, Heijunka not only finds defects in coatings, but also detects problems in fiberglass underlayment, color consistency, product wrapping, and even pallet counts. It can also continuously inspect in-process materials to ensure that specifications, such as dimensions, smoothness, straightness, and color, are met.

For this application, Heijunka would be looking at two production lines moving discrete boards at high speed.

• The primary line cuts large sheets of raw material to size. Cut sheets would need to be inspected for excess moisture as well as dents, debris, and scratches as small as a grain of rice. The customer runs hundreds of different product types through this conveyor, each being cut to a different size and configuration.
• The second line would be responsible for monitoring the quality of the milling process, specifically looking for chipping along the edges.

Machine vision analytics where they need to be

Intrinsic Imaging’s Heijunka Vision analytics-as-a-service software continuously inspects in-process materials to ensure that specifications such as dimensions, smoothness, straightness, and color are met. A manufacturer wanted to integrate this cloud-based machine vision solution directly into its process control system, rather than hook it to a SCADA system for quality control. Intrinsic engineers found Opto 22’s groov RIO edge I/O, which let the Heijunka software be hosted on AWS and publish MQTT messages to a hosted broker that were then bridged to an on-premises broker. The groov RIO edge I/O solution let Intrinsic support more of the last mile between cloud software and physical action, while letting the manufacturer speak the language they were most comfortable with.

However, unlike most Heijunka applications, the customer also wanted a PASS/FAIL I/O signal that it could integrate directly into the PLCs that would handle material rejection. By bypassing the SCADA and providing a direct path to PLC action, the customer hoped to simplify integration and reduce latency. However, Intrinsics hadn’t integrated its cloud-based software directly with a hardware system before.

“This was surprisingly difficult to figure out,” said Eric Cheng, CTO at Intrinsics. “I wasn’t sure what kind of device could provide a physical interface to our cloud-based system and also satisfy our technical and operating requirements.”

Besides needing a device that could tolerate an industrial environment and integrate with Heijunka’s existing software stack, Eric needed something that would operate with minimal latency. From the time a given video capture was sent to Heijunka, the customer would have a roughly five-second window in which to detect and reject a problematic part. Heijunka would need to return a pass or fail indication that consistently fell within that window of opportunity.

Finding the missing piece

According to Eric, Heijunka’s communication infrastructure is usually built on MQTT. “Generally, we set up an MQTT broker and publish to an agreed topic,” Eric explained. “It’s usually new for customers, but it’s typically the best way to meet their performance and security requirements. [MQTT is] lightweight and low-latency, there’s plenty of open-source support for it, and it doesn’t require us to reach into their network from the outside. Amazon Web Services (AWS) has MQTT managed services to host a broker, and in cases where we are hosting the system ourselves, it’s all [deployable] out of the box and in the cloud.”

In this case, Heijunka would be hosted on AWS and publish MQTT messages to a hosted broker, which would be bridged to an on-premises broker in the customer’s facility. (Bridging allows brokers to exchange data behind the scenes so that it’s available to clients on both systems.) This architecture proved to be the key factor in choosing groov RIO for the final piece of Intrinsics’ solution.

“The customer found [a similar edge I/O device, and it] made me realize that an MQTT device could work for what we were doing,” said Eric. “I started searching around . . . Groov RIO was exactly what we needed, and it was eye-opening to see everything we could do with it.”

groov RIO had the industrial build Eric needed, including a wide operating temperature range and UL Hazardous Locations and ATEX approvals. It was also compatible with his software stack. He said, “Groov RIO was on the same wavelength as us: built-in MQTT, Linux-based, web interface, and it just seemed more modern than some [other devices] that still require Windows 7 executables for configuration. I didn’t want to have those kinds of dependencies.”

Given the nature of the request, Eric’s customer was also interested in the cybersecurity of the proposed architecture and appreciated that the groov RIO modules included a network firewall.

Putting the cloud in control

Intrinsics built an isolated network to connect cameras and groov RIO MM1 modules (GRV-R7-MM1001-10) to the on-premises MQTT broker. A separate network connects that broker to the internet for video streaming to Heijunka Vision and data exchange with the hosted MQTT broker, both running on AWS.

The groov RIO MM1 module provides eight channels of universal I/O with support for more than a dozen software-selectable signal types. I/O data can be shared via MQTT, REST, VPN, or traditional protocols like Modbus/TCP. Communications are secured with user authentication, a local firewall, and TLS X.509 certificates.

In this case, each groov RIO module makes an encrypted connection to the local broker, which has only port 8883 open, the standard port for MQTT TLS connections. The decision to bridge between two MQTT brokers is also for security. The local broker acts as a firewall for the OT side of the system while still allowing groov RIO data to be exchanged with Heijunka in the cloud. “The goal is to keep the RIOs inaccessible from the outside,” said Eric.

To satisfy another customer request, each production line uses two groov RIO modules with each configured to provide eight discrete inputs (Figure 1). Production line PLCs encode the product ID for the specific part being examined by Heijunka as a 16-bit integer and send each bit to one of the inputs on the RIO pair. A Node-RED flow in each RIO module publishes its eight input channels as MQTT topics, which Heijunka combines to decode the product ID and select the appropriate set of algorithms for that product type.

The groov RIO modules also use Node-RED to subscribe to quality indicators, which Heijunka publishes to the MQTT broker. One of the relay outputs in each pair of modules is used to indicate the PASS/FAIL decision returned by Heijunka for a given part. The production PLCs watch these outputs and use them to trigger a physical rejection of the product if needed.

Since Heijunka performs all the heavy computation and product identification, the groov RIO modules can run the same logic without regard for the product type, creating a clean interface between cloud and edge networks.

Fast, automated quality control

At this point, the customer has been automatically rejecting defects for several months and plans to introduce Heijunka in the rest of its facilities. Intrinsics confirmed a round trip time, from measurement to result, of less than two seconds.

“I’m impressed with how fast it is even though we are taking two or three steps,” said Eric. “Most of that latency is due to transmitting video over the network.”

Intrinsics’ customer is using Heijunka to save on labor costs and increase quality with an overall goal of avoiding material returns. The customer can review system performance through Heijunka’s built-in trending, which displays metrics like the number of defects detected per hour. Over time, the customer can monitor historical trends in defect rates and diagnose the root cause of elevated defect levels. A user can click on any bar in the histogram and immediately bring up replay images to see every defect the system has ever detected. Each one of the defects in the plot above indicates a product that triggered a reject signal in Heijunka, which then made its way to the groov RIO modules via MQTT (Figure 2).

“We were under the gun to do this quickly,” said Eric, “but we got it figured out in less than a month . . . Now we can provide a direct physical interface to low-level automation systems. . . . Using the RIOs allowed us to own more of the ‘last mile’ between cloud software and physical action and allowed the customer to speak the language they were most comfortable with. That allowed a cleaner separation between our software expertise and their hardware expertise.”

With groov RIO, Heijunka can now be adapted to many more applications, supporting both hardware and software interfaces, whichever produces the best performance.

For more on Intrinsics Imaging, visit the website.

This case study originally appeared in Automation 2022: Cybersecurity & Connectivity Volume 2.

Read Ebook

---

---