Cutting-edge research in quality sorting

Advances in artificial intelligence could enable the detection of internal disorders using in-line X-ray imaging. By Anna Mouton.

“It takes a lot of effort to produce fruit at the right maturity and of good quality at harvest,” said Dr Pieter Verboven. “But things that happen during the growing season or the storage period could cause problems in the postharvest supply chain.”

Verboven was addressing attendees in a keynote presentation at the recent Hortgro Postharvest Symposium. He is the industrial research manager of the MeBioS Postharvest Group at KU Leuven in Belgium.

One of their focus areas is in-line sorting to eliminate suboptimal fruit before it reaches the consumer. Verboven shared their progress in applying artificial intelligence to identify postharvest damage and disorders on X-ray images.

Seeing inside fruit

Machine vision is already widely employed for identifying external defects on the pack line. “Where things are still challenging is when we talk about internal quality,” said Verboven, “especially defects that are not visible from the outside.”

Assessing internal fruit quality often comes down to cutting open a sample of fruit from each batch. If a problem is detected, the entire batch could be discarded even though some of the fruit are sound.

The ideal sorting technology would allow non-destructive internal examination of individual fruit. Such technology must be accurate, cost-effective, and fast enough to implement on a commercial pack line.

While in-line sorting based on near-infrared imaging is already available, it requires onerous calibration, and Verboven thinks it might not be robust. His group has been working on X-ray imaging as an alternative technology for seeing inside fruit.

“The properties of X-rays are beneficial for this task,” he said. “They are electromagnetic waves with very short wavelengths, so they can penetrate materials. You can detect things that are deeper inside the fruit than just the surface layers.”

X-rays are well-established in medical applications and material sciences. “We see it used more and more in the food industry on quality-inspection lines,” notes Verboven, “It can detect foreign objects in packages to prevent unwanted materials from going into the supply chain.”

Different dimensions

A single X-ray image is generated by placing an object between an X-ray source and a detector. The object partially absorbs the X-rays, while those that pass through strike the detector, creating an image corresponding to the object’s internal structure.

Conventional X-ray images are two-dimensional representations of three-dimensional objects. Computed tomography combines X-ray images taken from different angles to produce a three-dimensional picture.

“Tomography gives you the details of the internal structure much better,” said Verboven. “You start to see all sorts of gradients within the fruit structure, which is what we’re after.”

For example, his laboratory has used X-ray computed tomography to track the development of internal browning in individual Braeburn apples over several months of storage. Although this technology could theoretically be installed on a pack line, it has two significant drawbacks: an eight-digit price tag and a several-minutes-per-image speed.

“A cheaper option would be a conveyor belt with a simple X-ray source and detector, producing 2-D pictures,” said Verboven. “This would be much more cost-effective and would be compatible with the speeds we need on sorting lines.”

He presented two- and three-dimensional images of a pear infested with a codling moth larva to illustrate the different technologies. Both clearly show the feeding damage. He contrasted this with a two-dimensional image of internal browning (Figure 1).

“Here it’s more challenging to see which is the healthy and which is the defective fruit,” he commented. “That’s where artificial intelligence comes into the picture.”

Researchers teach machines

“Artificial intelligence is a broad term that covers all kinds of computer algorithms that make decisions for us,” said Verboven. “We can inspect images ourselves, but we should teach computers to decide which fruit are defective without our intervention.”

Machine learning is one type of artificial intelligence. Verboven explains that it involves processing and analysing images to obtain properties called features. For example, fruit density could be a feature.

A single image can have hundreds of features, and some of these will be relevant for classifying the image. For example, low-density areas could be a relevant feature when classifying the presence or absence of codling moth damage.

Feature-based detection typically involves image manipulation to remove noise and accentuate features. A human dictates how images should be manipulated, which features are relevant, and the threshold for each feature.

Verboven used the images of the codling moth-infested pears to demonstrate the process. His group was able to teach the machines to classify pears as infested or uninfested with an accuracy of 56%–68%.

“This is not great,” he admitted. “I assume you would not accept that as an industry.”

Machines teach themselves

“We run into limitations when using conventional machine learning for these kinds of problems,” said Verboven. “Another method that can be used instead is deep learning, a newer kind of artificial intelligence that is replacing machine learning.”

Whereas conventional machine learning involves a human teaching the machine how to classify images, deep learning lets the machine figure it out for itself by providing a training set of many pre-classified images.

“The deep-learning algorithm will do millions of operations on these images, many more than we can even imagine,” explained Verboven. “In the end, it will also generate a feature system to classify the images.”

His group achieved an accuracy of 96% by using open-source deep-learning algorithms to classify infested and uninfested fruit. “The deep-learning method outperforms conventional machine learning across all stages of infestations,” he said.

In future, the high accuracy of deep-learning algorithms combined with cost-effective and rapid two-dimensional X-ray imaging could revolutionise in-line sorting. However, Verboven had some caveats.

“We need proper validation and testing of these methods before we deploy them,” he said. “And we still need the industry experts to help us choose the right methods, so we need to do this with you.”

Watch Verboven’s full presentation on the Hortgro YouTube channel.