Issue in AEM Instance showing High CPU Usage | Community
Skip to main content
Hemalatha_Bodab
Level 2
Sep 23, 2019 10:00 AM UTC
Solved

Issue in AEM Instance showing High CPU Usage

Hi All,

We are frequently experiencing the High CPU usage Issue with the AEM instances. Please find the Below Description.

1.We have Production Environment where we configured 4 Publish Instances and 2 Author Instances.

2.We configured AEM on App nodes.

3.Frequently ,we are Experiencing High CPU usage Load i.e CPU usage is Hitting 98% on one of the Publish instances and remaining for 10 hours nearly.

Also,at the same time we are not seeing any load on other Publish Instances.

4.We verified the Dispatcher level configurations and logs also,we are seeing normal request processing and there are no errors in Error logs.

5.While Analyzing the Threads Dumps for  that Time Instance we are seeing below threads in blocked state.

Thread dump 6/10
"ApacheSlingdefault_QuartzSchedulerThread" prio=5 tid=0x298 nid=0xffffffff waiting for monitor entry
     java.lang.Thread.State: BLOCKED
     at java.lang.Object.wait(Native Method)
     - waiting to lock <0x39d83e0e> (a java.lang.Object) owned by "null" tid=0x-1
     at org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:311)

Thread dump 9/10

"sling-default-28" prio=5 tid=0x104d nid=0xffffffff waiting for monitor entry
     java.lang.Thread.State: BLOCKED
     at sun.misc.Unsafe.park(Native Method)
     - waiting to lock <0x2a446ba5> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) owned by "null" tid=0x-1
     at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
     at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
     at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
     at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)


Could anyone please Suggest any solution for this issue.

    This post is no longer active and is closed to new replies. Need help? Start a new post to ask your question.
    joerghoh
    joerghohAccepted solution
    Employee
    Sep 25, 2019 5:37 PM UTC

    Hi,

    Short answer: these threads are not blocking anything or are not blocked by anything. This is just the implementation of the Object.wait() method, which manifests itself in this way.

    You can essentially ignore these 2.

    View reply

    11 replies

    adithyaa4585051
    adithyaa4585051
    Level 2
    Aug 14, 2020 4:19 AM UTC

    Hello Hema,

     

    We are also facing the same issue, if you are able to fix it can you please let me know.

     

    Thanks,
    Adithya

    Translate
    aemmarc2
    Employee
    Sep 27, 2019 1:35 PM UTC

    This thread tells you nothing.

    You're looking at the wrong things. Look for what's running.

    java.lang.Thread.State: RUNNABLE

    Translate
    joerghoh
    joerghoh
    Employee
    Sep 26, 2019 5:15 PM UTC

    This is a simple threadpool thread waiting for a job...

    Translate
    Peter_Puzanovs
    Peter_Puzanovs
    New Participant
    Sep 26, 2019 1:32 PM UTC

    Hi Hema,

    Have you looked into: /system/console/status-slingscheduler is there any job that's running when CPU is high?

    Possibly there is a job that concurs with high load?

    Regards,

    Peter

    Translate
    Hemalatha_BodabAuthor
    Level 2
    Sep 26, 2019 1:17 PM UTC

    Yes .We able to see numerous threads with 0x2a446ba5.Please find below snippet and let me know your findings

    "sling-default-50" prio=5 tid=0x1109 nid=0xffffffff in Object.wait()

       java.lang.Thread.State: WAITING (on object monitor)

    at sun.misc.Unsafe.park(Native Method)

    - waiting to lock <0x2a446ba5> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) owned by "null" tid=0x-1

    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)

    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)

    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)

    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)

    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)

    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

    at java.lang.Thread.run(Thread.java:745)

    Translate
    joerghoh
    joerghohAccepted solution
    Employee
    Sep 25, 2019 5:37 PM UTC

    Hi,

    Short answer: these threads are not blocking anything or are not blocked by anything. This is just the implementation of the Object.wait() method, which manifests itself in this way.

    You can essentially ignore these 2.

    Translate
    sdang
    Employee
    Sep 25, 2019 1:26 AM UTC

    Can you find within the thread dumps to see which thread is holding onto this lock "0x2a446ba5"? This would tell you which thread is blocking this thread in question.

    Translate
    aemmarc2
    Employee
    Sep 24, 2019 1:18 PM UTC

    Careful using the word "deadlock" as that means something very specific.  I haven't seen a true case of deadlock in aem in quite some time.

    Translate
      aemmarc2
      Employee
      Sep 24, 2019 1:13 PM UTC

      BLOCKED threads are symptoms of a problem not the problem itself.

      You need to look at the RUNNABLE threads over a series of thread dump captures to understand what's causing resource contention.

      Also use a better way to capture the thread dumps (like the Jstack series script @JaideepBrar suggested). Whenever you see the TID as 0xffffffff it implies you used the same JVM thats buggered to capture the thread dumps. You need to launch jstack using a separate jvm than the one running AEM.

      Translate
      Hamid1350
      Employee
      Sep 23, 2019 1:25 PM UTC

      Usually a deadlock could cause high CPU usage. You can check the following document where you can find info on how to analyze sloe and blocked processes:  Analyze slow and blocked processes

      Translate
      Show more replies
      Terms & ConditionsAccessibility statement

      Loading footer...