Solution to clear an Oracle Enterprise Manager Ops Center stuck job

Solution Notes

• The solution below should work with any stuck job not willing to disappear, i.e. if you try stopping the job from the UI or CLI and its not working.
• The issue and solution below are even for the latest Oracle Enterprise Manager Ops Center 13c.

We performed an upgrade from 11g to 12c. After successful upgrade we have this problem:
Before the upgrade, we had two jobs in status “Stopping” which did not stop since 20 days. These jobs should disappear after the upgrade.
However, they are still there and prevent that other jobs for the same asset can be completed, successfully.

How can we delete those jobs from the database?

We get the following errors when using the CLI:

# cd /opt/SUNWoccli/bin
# ./oc

    Oracle Enterprise Manager Ops Center
      Copyright (c) 2007, 2012 Oracle and/or its affiliates. All rights reserved.
      Use is subject to license terms.

      Use the -connect- command to get started
      The tab key will always show the available command set
      
xvmSh > connect
localhost > jobs
localhost/jobs >

delete list rerun_on_failed run stop

For example if you try to do this from commend line, you get the below error.

localhost/jobs > delete --id hostname.759
    delete of job failed: Exception in Deleting Job(hostname.759) - Job is in progress. Cannot delete this job. Please stop the job and then delete.. Refer EC cacao logs for more details.

localhost/jobs > stop --id hostname.759
    stop of job failed: Exception in stopJob for JobId(hostname.759) - User:root does not have permission to run task 'DeleteSvcTagsTask' with permissions{READ} on object:__satellite/com.sun.hss.domain:name="urn:st:005f2b98-65f8-e53c-ccd3-ae25f1699f99",type=ServiceTag. Refer EC cacao logs for more details.

localhost/jobs > stop -f --id hostname.759
    stop of job failed: Exception in stopJob for JobId(hostname.759) - User:root does not have permission to run task 'DeleteSvcTagsTask' with permissions{READ} on object:__satellite/com.sun.hss.domain:name="urn:st:005f2b98-65f8-e53c-ccd3-ae25f1699f99",type=ServiceTag. Refer EC cacao logs for more details.
    localhost/jobs > exit
#

The solution to the issue is, to mark the job as done in the database, follow the steps below to clear the job.

First, we need to get the job run_id. run to below to see that.
Note: Please replace 759 with your Job_id in the examples below.

SQL> select * from scn_job_run where Job_id=759;

    JOB_ID     RUN_ID
---------- ----------
START_TIME
---------------------------------------------------------------------------
END_TIME
---------------------------------------------------------------------------
    STATUS DIAGN
---------- -----
        759          0
19-JAN-17 11.44.53.735000 AM
29-MAY-17 04.47.56.686000 PM
        21

In the example above, the job run_id is 0.

Next, lets change the job status.

ecadm sqlplus

SQL> update scn_job_run set status=12 where Job_id=759 and run_id=0;
...
SQL>commit;

In some instances, setting the job status to 12 will not help.
In such a case try setting the job id to 16.

Follow the below to do that.

./ecadm sqlplus

SQL>update scn_job_run set status=16 where Job_id=759 and run_id=1;
SQL>commit;

Once committed to the database, you shuld be good to go, just refresh the Ops Center UI, or CLI, and the stuck job will be gone.

Source: Oracle Doc ID 1513641.1