The reported scenario is:
- UDM jobs are stuck in "In Progress" status and their step is stuck in "Queued" state.
- In the AI job there is a transformation which contains 3 steps:
       1. Excel Input.
       2. Get variable.
       3. AR OutPut to the staging form.

When running the DMT (UDM) Job, it remains in "In Progress" and it's step is getting stuck at "Queued".

Checked further by performing these steps too:
1. Clear out the records from DMT:SYS:SequencingEngine form with Parent_Job_GUID = "DO NOT REMOVE" .
2. Rebuild the Sequence table from this form - DMT:SYS:StagingFormDependency .
3. Found one entry for each job in CAI:Events form where status was WARNING .
4. Clear the records from DLD:ThreadManager form.
5. Observed Source and Target Server Name field values in CAI:Events form which holds app load balancer name.
 

This knowledge article may contain information that does not apply to version 21.05 or later which runs in a container environment. Please refer to Article Number 000385088 for more information about troubleshooting BMC products in containers.

Checking the arjavaplugin.log it was observed multiple errors related to the connectivity from the main Java plugin server.

Example:
................................
...etc...
ERROR (90): Cannot establish a network connection to the AR System server; Connection timed out: connect <{LOAD-BALANCER NAME}>:<{PORT}>
    at com.bmc.arsys.apitransport.ApiProxyJRpcBase.connectionTry(ApiProxyJRpcBase.java:136)
    at com.bmc.arsys.api.ProxyJRpc.getRpcClient(ProxyJRpc.java:148)
    at com.bmc.arsys.api.ProxyJRpc.<init>(ProxyJRpc.java:67)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
    at java.lang.reflect.Constructor.newInstance(Unknown Source)
.. etc....
................................

Steps to perform:
1 Create an entry in the UDM:RAppPassword form containing the Load Balancer name. See RApp Password form for more information
2 Check and if required add in the Hosts file in the OS and entry containing the IP address of the Load Balancer (Short name and FQDN)
    Note: If the environment is a Server Group edit the Hosts file on all servers' OS of the group.
3 Restart the AR System application service and run the job again.



Additional article(s):
Remedy - Server - Error:"ERROR (91): RPC call failed; {AR servername}:{AR Port} Connection reset" on CAI:Events running a UDM job