Fredhopper Indexer's qserver instance consuming too much CPU

Question

Preview page is loading within 1.7 mins, FH query took 6.084 secs and QServer consumes more than %95 CPU. This as a blocker problem, results in slowing down the whole system. Is there any way of identifying the problem and the reason why it cannot get any better ?

I am using the indexer for both indexer and qserver instances.

The actions I've taken and did not help are;

1. Increasing qserver memory of indexer from 1500 to 6000; instance3_qserver_memory=6000

2. We started the procedure of reindexing . We stopped the index server , took a backup of fredhopper with ; bin/capture-import <INSTANCE> path/to/capture.zip

   We copied all of the files in

   <BASE_DIRECTORY>\INDEXER>\data\xml\processed\batch; into <BASE_DIRECTORY>\<INDEXER>\data\fas-xml\<CATALOG01> .

   We ran the "reindexer" with; bin/reindex indexer command.

   And also run the following command to update the configured live servers with the current index and configuration; bin\fresh-index-to-live indexer

3. We observed that the items are being indexed, and the number of items being indexed is being counted down. When there is no item left to index, then I restarted the qserver of indexer with the following command.

bin\deployment-agent-client --location localhost invoke <INDEXER> qserver stop bin\deployment-agent-client --location localhost invoke <INDEXER> qserver start

4. None helped. Each time I check the processes, Dapp-name=qserver instance consuming too much CPU up to 350%. I observed the kitchen, indexer instance logs, Fredhopper.log and qserver.log files. What I see is the following (I doubt they cause to excessive amount of CPU usage),

qserver.log files "http-nio-10180-exec-39" daemon prio=10 tid=0x0000xxxxxxx nid=0x4e27 runnable [0x00007fb8909eaxxx] java.lang.Thread.State: RUNNABLE Locked ownable synchronizers: - <0x00000007e1280xxx> (a java.util.concurrent.ThreadPoolExecutor$Worker)

fredhopper.log 2016-01-05 17:58:43,834 WARN [http-nio-10181-exec-32] SortFieldFilter.java:38 50180 Removing the field 'pubtitle' from the sort fields. Sorting on non-live attributes is not supported. Please include this field in the tree attributes in order to sort on it. || fredhopper@hostName:fredhopper/data/installations/FAS tags/fas/fas-7.5.0.16/engine@107262 - url: http://{url}

Since it states that "Sorting on non-live attributes is not supported." , I go to the Business manager and checked the pubtitle attribute if it is live or not. And I confirm it is live . So why is it throwing such SortFieldFilter WARN, no clue !

EDIT : The exception I gave in qserver.log, is thrown thousands of times; could the excessive usage of CPU be caused by this problem ?

There is an entry regarding the exception;

EDIT 2: I also checked the CPU usage in SIT, and observed that its behaviour in terms of CPU usage is similar to the UAT's . In another word, overall CPU usage is up to 80%, and qserver instance's CPU usage is around 250%. So, there is a similar behaviour, but SIT can handle the operations with no problem. At least for now.

Answer 1

Did you analyze your relevant code and ensure there is no scope of memory leak?

May be you can try replicating the environment on a separate server - may be Dev and try reproducing the scenario there. Further you can remove all possible integration points to the fredhopper server and try checking with simple queries on it.

I hope it may help.