Virtual assistance right in front of your eyes – how to combine augmented reality and artificial intelligence

From order picking to personal healthcare assistance and support for after-sales service work: computer technology worn on the body or the head is becoming increasingly important in a wide range of sectors. One of the global leaders in the provision of these wearable computing solutions is Bremen software company Ubimax, founded in 2014.

Retrieving goods from a warehouse and assembling them into a shipment – what logistics calls ‘order picking’ – is often a very monotonous and error-prone task. Armed with a list of items and a hand scanner, and always pressed for time, pickers have to find the right shelf in the warehouse and then put together the various customer orders correctly. That is why the Coca-Cola Hellenic Bottling Company, based in Zug in Switzerland, one of the drinks manufacturer’s leading bottling plants, began to take a new approach last year. “In order to drastically reduce misdeliveries, the company’s 400 pickers are now working with our software solution xPick and achieving a quality rate of 99.99 per cent – that’s virtually perfect,” reports Dr Hendrik Witt, the CEO of Ubimax. “The employees are assisted in their work by smart glasses with a transparent display, which utilise graphics to direct them straight to the storage area they require and then use the built-in camera to scan the barcode as well.” Another benefit is that both hands remain free throughout, so that items can be picked up quickly and easily to add them to the picker’s goods pallet.

Fully integrated solution platform: Ubimax Frontline

Ubimax first introduced its xPick picking solution in 2014, and has updated it several times since then. Now the tool has been included into the integrated Ubimax Frontline platform, which also contains the xMake assembly assistant and quality assurance, the xInspect service and maintenance tool, and the xAssist remote support solution, which is much in demand in the coronavirus era. “This is the world’s first fully-integrated solution platform that covers the entire value chain, and our aim is to revolutionise the day-to-day work of non-desk-based employees – much like Microsoft Office did for office workers,” explains Hendrik Witt. “Ubimax Frontline doesn’t just increase employees’ productivity, it also helps to make their actual work more satisfying.”

A steadily expanding product range, a 400-strong customer base, and a team currently consisting of some 100 specialists located around the world – in Bremen, southern Germany, the US and Mexico – are all testament to Ubimax’s impressive growth. In October 2019, the firm acquired ESSERT Digital, one of Europe’s leading companies for augmented-reality-based remote after-sales service solutions, located in Ubstadt-Weiher in Baden-Württemberg. Ubimax has included the AS 2.0 remote support solution in its Frontline platform, providing it with another remote service tool in addition to xAssist.

Comprehensive deployment: at all 400 BMW workshops in the US

xAssist is linked to customers’ existing workshop management systems using a standardised interface and can then be configured individually. “One of our major customers is the BMW automotive group, which has deployed the software across all of its 400 workshops in the US,” says Hendrik Witt. “Our smart glasses technology allows workshop employees on the ground and a remote expert to see exactly the same thing, making it easy to give precise instructions and resolve problems quickly. At the same time, or later on, the recordings can also be used to train new employees, for documentation purposes, or for quality assurance.”

Application of artificial intelligence

The various Ubimax solutions do more than just simplify processes: they can also use artificial intelligence to detect irregularities in daily procedures. “That could be an incorrect posture during certain activities – which could have health and safety implications,” Hendrik Witt explains. “Or the software might show that the wrong tool is being used for a particular task and correct this.”

The company’s innovative approach has also been recognised by various awards, such as ‘The Spark – The German Digital Prize’, which Ubimax received at the end of 2019. That award is for companies who are implementing innovative concepts to optimise the everyday working environment. Its focus is on augmented and virtual reality, voice and gesture control, and on innovative methods for virtual training.

A success story made in Bremen

Software specialist Ubimax is proud of such awards, of course. And the state of Bremen has every reason to be proud as well, as home to one of the world’s leading companies in the field of wearable computing solutions. “When we first started out, we received innovation support from Bremen for two of our projects,” remembers Hendrik Witt. They used the research, development and innovation programme (FEI) of BAB, the development bank for Bremen and Bremerhaven, which helps businesses who are developing new technologies to minimise risk and achieve technological breakthroughs.

Witt thinks that more could be done locally to provide follow-up growth finance, but that Bremen is otherwise well positioned when it comes to AI, not least through its universities. “And that is a huge benefit for us, of course, since we are always on the lookout for skilled specialist staff. Not that we wouldn’t employ people from outside Bremen.”

Ubimax is generally fairly optimistic about the future, despite the current coronavirus crisis. “At least we have a presence across all industries, which makes us very robust overall,” is Hendrik Witt’s assessment of the company’s situation. “On top of that, our remote support emergency kit offers useful standard equipment that allows users to control a wide range of processes remotely, so companies are well prepared to deal with any eventuality. Regardless of the current situation, I remain absolutely convinced that wearables are the up and coming thing. At the moment, the technology is almost exclusively used in B2B, much like the mobile phone was back in the day. But I reckon that it’s only a matter of time until it starts to become a consumer accessory as well. In the medium term, it’s entirely conceivable that we may get involved in that area too.”