2

We are trying to implement Search module in SDL DXA 2 and integrate it with the elastic search. When we publish a page which has binary content, from the logs we could find that the binary indexing was taking place first and after that the page content.

  1. Why is binary indexing taking place before the page indexing?
  2. Also from the binary indexing process is there a way I can get the page id of the actual page it is residing because I need this info as well for indexing.
  3. While indexing the page is there any way to get the embedded schema components? For eg: if my page has a component with text fields, date, embedded media schema etc, right now page indexing only provides the text from all these data fields clubbed together. Is there a way I can get other fields like the date, embedded media etc so that I can call the custom methods for binary indexing from here itself?
  4. Is there any other way to get the media files like pdf or it's component id from the page index?

Log files: We can see that binary indexing is taking place 1st and then page indexing

2018-03-15 13:58:22,637 INFO ElasticIndexer - Adding binaries to elastic.

2018-03-15 13:58:22,637 DEBUG ElasticIndexer - Dispatching binary content to elastic.

2018-03-15 13:58:22,637 INFO ElasticIndexer - Indexing binary with Id: binary:5-4269, and URL Path:/xxx/xxx.pdf

2018-03-15 13:58:22,637 INFO ElasticIndexer - data.getPublicationItemId() 5

2018-03-15 13:58:22,637 INFO ElasticIndexer - data.getIndexUrl() /xxx/xxx.pdf

2018-03-15 13:58:22,637 INFO ElasticIndexer - data.getFileSize() 42025

2018-03-15 13:58:22,637 INFO ElasticIndexer - data.getFileType() pdf

2018-03-15 13:58:22,637 INFO ElasticIndexer - data.getFileName() xxx.pdf

2018-03-15 13:58:25,673 TRACE ElasticIndexer - inside processItemUpdates elastic

2018-03-15 13:58:25,674 INFO ElasticIndexer - Adding pages and component presentations to elastic in batches of 10

2018-03-15 13:58:25,678 INFO ElasticIndexer - Adding Id: tcm:5-304-64

2018-03-15 13:58:25,678 TRACE ElasticIndexer - Adding: summary: xxx.

2018-03-15 13:58:25,678 TRACE ElasticIndexer - Adding: parentsgid: 66

2018-03-15 13:58:25,678 TRACE ElasticIndexer - Adding: itemtype: 64

2018-03-15 13:58:25,679 TRACE ElasticIndexer - Adding: schemaid: 62

2018-03-15 13:58:25,679 TRACE ElasticIndexer - Adding: sgid: 66

2018-03-15 13:58:25,679 TRACE ElasticIndexer - Adding: sgid: 63

2018-03-15 13:58:25,679 TRACE ElasticIndexer - Adding: sgid: 4

2018-03-15 13:58:25,679 TRACE ElasticIndexer - Adding: title: xxx

2018-03-15 13:58:25,679 TRACE ElasticIndexer - Adding: publicationid: 5

2018-03-15 13:58:25,679 TRACE ElasticIndexer - Adding: type: 0

2018-03-15 13:58:25,680 TRACE ElasticIndexer - Adding: body: xxx

2018-03-15 13:58:25,680 TRACE ElasticIndexer - Adding: url: /xxx/xxx/xxx.html

2018-03-15 13:58:25,680 INFO ElasticIndexer - Dispatching documents in 1 steps.

2018-03-15 13:58:25,682 INFO ElasticIndexer - Clearing out registers.

This is how the indexing XML data looks like:

<indexdata>
<url>/xxx/xxx/xxx.html</url>
<title>xxx</title>
<publicationid>5</publicationid>
<schemaid>62</schemaid>
<itemtype>64</itemtype>
<parentsgid>66</parentsgid>
<sgid>66</sgid>
<sgid>63</sgid>
<sgid>4</sgid>
<type>0</type>
<body>xxx</body>
<custom>
    <summary>xxx</summary>
</custom>
</indexdata>
Improve this question
4
  • you can able to include custom fields IN Filed mapping and Link Fields to Embed in the TBB, then you can able to see those fields data in the custom section in the indexing XML
    – Velmurugan ♦
    Commented Mar 15, 2018 at 15:37
  • Please edit your questions to add a bit more detail about what you are trying to do exactly with search functionality Is there any specific reason looking to index the binaries used on the pages?
    – Velmurugan ♦
    Commented Mar 17, 2018 at 9:58
  • Yes, we are trying to index the binaries used on the pages. While doing so we are not getting the pageid where the binary resides.
    – Sreelakshmi Viswam
    Commented Mar 18, 2018 at 6:00
  • @Velmurugan to answer your query the business requirement is that if the search term is found in some PDF files or other documents then the search results page should display the title and link of the page referencing the PDF (in addition to showing the PDF and its link).
    – Husain Zahid
    Commented Mar 18, 2018 at 11:04

1 Answer 1

Reset to default
1

An answer to question1:

AFAIK - Based on the transaction first will fire the dependencies used on the page and then page object will process to commit the changes.

But It doesn't mean that it will be indexing the same way, it can be customized.

If you look at the SolrIndexer.java in the commit method, It's indexing in the following orders.

this.commitAddContentToSolr(this.itemAdds);
this.commitAddBinariesToSolr();
this.removeItemsFromSolr();
this.processItemUpdates();

An answer to question2:

I'm afraid, It's not possible. I'm not sure it's possible to get out-of-box by backtracking based on binary item-reference-id to get the component Id to find out the page Id.

An answer to question3:

In the Generate Index Data TBB you can able to include custom fields In Filed mapping and Link Fields to Embed in the TBB, then you can able to see those fields data in the custom section in the indexing XML.

Regarding binary indexing on elastic search, I'm not sure about Is it supported for binary content searchable.

Binary datatype - In elastic search

The binary type accepts a binary value as a Base64 encoded string. The field is not stored by default and is not searchable:

I hope it helps.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.