Will Semantic AI Turbo Charge RPA Software Robots?
“Its Automation, Jim, but not as we know it” to paraphrase from Star Trek.
The potential to create the ultimate in Automation or just Ultra Hype?
Could Semantic AI dramatically boost the investment many have already made in RPA software robots by cutting the maintenance effort?
What is Semantic AI? Just ChatGPT on Steroids?
There is no official definition of Semantic AI, it is generally regarded as the combination of technologies namely Resource Description Framework (RDF), Ontologies, SPARQL (pronounced “sparkle”, RDF Query language), web technologies and AI.
The aim of Semantic AI is to use semantics to enable AI code to better understand, interpret and use reason with large collections of data.
ChatGPT is a generative AI product that uses a Large Language Model (LLM) to determine the construction of an answer in English using known patterns and probability to a question.
Although these two approaches are slightly different, the potential combination offers significant benefits as it can increase the accuracy of results and make better use of context.
Generating Automations from text prompt?
Start with a text prompt to generate an automation. It sounds easy. There are plenty of examples where Generative AI is used to generate code, it is leveraging its knowledge of code that it was trained on, LLM techniques to produce code that delivers the functionality requested in the prompt.
Of course, code generation is not new. There have been previous waves of hype about Third Generation Languages (3GL), Fourth Generation Languages (4GL), use of templates, code generators, etc.
The complexity of modern computing tech stacks used in the development, test and production environments to deliver load balanced, resilient, secure functionality at scale, means that the construction of the code is only one part of an IT solution.
There has been lots of research on requirements specification and many law suits on the interpretation of the words into code. AI always has a Probability factor with any result – but that value is not shown by ChatGPT and its competitors which can give the impression that it is 100% accurate.
The explosion in the number of “Cheat Sheets” demonstrates that the creation of effective prompts for Generative AI is not easy.
In the short term, some of the Generative AI technologies will be trained on more automation code and therefore will be capable of generating more effective Automation solutions which can be rapidly deployed for testing and execution in Automation architectures.
Is it Computer Vision with Good Glasses?
One of the challenges with Automation and in particular RPA (Robotic Process Automation) is the way the technology works with existing GUI application displays.
UiPath helped lead the RPA industry with its “Selector” approach and its capability to work with VDI (Virtual Display Images) with key products such as Citrix.
Similarly, UiPath has been a leader with Computer Vision, which is a way to enhance the resilience of an RPA automation to work with an application by more accurately being able to determine display elements than was possible with selectors. It is using AI functionality to determine the contents of the display.
The UiPath Computer Vision initially required the transmission of the display to an AI service hosted in the cloud that used GPU power to deliver the result. There is now also an option to use local CPU power available to the Robots to perform a similar interpretation of the display. This option avoids the security and processing complexity of the displays being transferred to and from the cloud.
RPA Software Robots with Machine Learning
Some AI specialists will say use Machine Learning so that RPA Software Robots can learn from errors and resolve any issues.
It is clearly correct that Machine Learning can be a mechanism to enhance the processing of RPA software robots. Machine Learning is about taking some AI functionality that has been trained on a set of data so that result is created and “Enhancing” the trained functionality to recognise an additional set of data.
The training of AI takes a lot of data, for machine learning to “Enhance” the result it also takes a quantity of data.
Why is “one Example” not enough for Machine Learning, the answer is Statistics.
The training is creating a pattern. For consistency of results, those statistics will tend to ignore “odd” results that do not fit the pattern.
For the Machine Learning to take effect it needs to have enough data to amend the pattern.
In a scenario where an existing RPA task starts to fail due to a change in a GUI display of an application. Machine Learning is not going to instantly fix the situation.
A person can intervene and “Show” the RPA robot how to work with the amended display. That knowledge could be captured by Machine Learning and as outlined above, if the learning is repeated sufficiently the Robot will be able to resume processing successfully without requiring manual support.
When an RPA Robot is not able to handle the application display change, it is effectively broken. Usually the situation is not dealt with using Machine Learning but is addressed by amending the code or updating the Computer Vision view of the display. It requires activity by an RPA developer to implement the fix.
Business Benefit – Lower Maintenance!
Semantic AI in combination with RPA Robots offers the possibility for the technology to detect any change in application display and determine how to resolve the difference so that the task can still be completed.
The difference is Semantic AI is likely to be able to determine whether GUI display is still in the correct context and adjust accordingly without the need for Machine Learning to enhance the original training data.
Semantic AI is not going to be magic.
If there is quantum change in an application, the knowledge of semantics to understand the context of the display will not be sufficient to overcome the change but it could cover a greater number of minor changes that application displays often experience.
By reducing the frequency of breaks in RPA robot processing of GUI displays, it will reduce the maintenance requirements and further improve the ROI for Automation.
Lots of complex ideas, academic theory and technical jargon but the key business benefit is Lower Maintenance costs.
Is Semantic AI Available Today?
UiPath have leveraged their extensive AI skills and technical knowledge of document understanding plus communication message understanding to create the “Ultimate Copy & Paste” functionality with “ClipBoard.AI”. It is in Beta / Early release.
Clearly as the Semantic AI type capabilities of “ClipBoard.AI” become proven the technology will be integrated with the rest of the UiPath automation capabilities.
There can be debate about definitions of Semantic AI, but practical use cases that can be easily deployed within existing business activity will deliver real benefits.
#businessbeyondautomation
Article Author
David Martin
Managing Director, Ether Solutions
https://www.ether-solutions.co.uk/
