Encoding problems are almost always about telling lies, albeit sometimes lies of omission. What I mean by this is that when sending data, you do three things:
Transform your data into a stream of bytes using an encoding,
Transmit the stream of bytes.
Tell your client what encoding needs to be used to interpret the bytes.
Sometimes you don't actually ...
I think in Solr 4.8 binary extraction changed, meaning a lot of the dependencies could be removed from deployer - this may simplify your setup. I recently experimented with Solr 4.10 (albeit with Tridion 2013) and found only the following additional jars were required for an SI4T deployer, all of which (with the exception of the si4t jars) could be found in ...
Spot on with your analysis Neil. I tried it out myself just now and experienced the same. The best resolution is to update that conditional logic to be:
&& (flaggedDcps == null || !flaggedDcps.Contains(id)))
I just updated the master branch on Github to reflect this.
Is your source encoding UTF-8? If so you should run the JVM with the argument
This can be achieved either by adding this parameter to the jvm.xml (if using a .Net web application) or in the startup parameters of your application container.
For jvm.xml (located in bin/config of your web application) the syntax is
SI4T requires a couple of fields to be configured in Solr's schema.xml in order index content coming from the Tridion CM and the SI4T TBB:
Fields that always need to be present:
id : the unique id of the individual item
url : the url of the published item
pubdate: the last publish date of the item
Additional fields coming from the SI4T TBB
Then, a ...
Solr moves very fast indeed. Yesterday version 4.6.0 was released and its quite likely that more minor version updates will be coming soon.
The good thing is that the actual library that SI4T uses to index content in SOLR, which is called SolrJ, always guarantees backward and cross-version compatibility for Solr's minor version updates, as is stated here.
tl;dr to Dominic Cronin's answer, encoding issues in SDL Tridion are always an issue of some system down the publishing/deployment line not being correctly configured.
Read Elena Serghie's blog post 7 Clues to Solve Character Encoding Issues for more details. My initial guess on what is wrong, will be the Deployer JVM settings (as these are by default not ...
In theory this should be possible. Note that you can only have 1 Indexer per storage configuration, so if you want to do this, you need something like:
<?xml version="1.0" encoding="UTF-8"?>
Indeed you can have different cores per publication, without different storage elements. The example config file illustrates this:
<!-- SI4T: The Value ...
Thanks for the documentation Raimond. It was very helpful.
Initially we faced some problems, but after removing few jar files and config updates it worked. This is now really working well with Solr 4.3.1. I really appreciate your work guys, this helped us solve many problems.
You have to do the following things to make it work with SDL Tridion 2013
Don't worry! The standard behaviour is that when you unpublish an item it will also be removed from the index. The functionality to remove from the index by republishing, is to enable users to un-index pages that are still needed on the site without having to unpublish then republish them.
I updated the documentation, to make this clear.
To prevent your other components being indexed you need to add a NoIndex metadata field to your schema. If the value in this field is "Yes" then the components will not be indexed. Note this also includes when you publish them directly - Im not sure if this is what you need...
Having just checked the source code, I see now that there is not ...
While I haven't successfully indexed any data just yet (see "SI4T: Commit failed after getSolrServer"), I did want to post the solution that got me past this initial hurdle. I'll post a more refined answer that includes a full list of the JARs I used along with which list I got them from in the SI4T Github repo once I've resolved my current issue.
For now, ...
Coincidentally I have just completed a proof of concept doing exactly this last week. My client is in the process of moving to Microsoft Azure, and replacing out existing GSA (Google Search Appliance) with Azure Search. They also have a hard “No Java Development” requirement.
The POC makes some use of Si4T by using the TBBs which generate the indexing data ...
The SI4T architecture is designed for this kind of integration, and is a much more solid approach than using TBBs (you cannot, for example control unpublishing from a TBB) or Event System (you cannot make indexing and publishing part of the same transaction from events).
If you are more comfortable working with .NET you could write the integration as a .NET ...
The downloads page contains PDF multimedia components, so I suspect this is the issue. When setting up Solr to index PDFs there are a lot of additional configuration settings and dependent libraries to add in. Double check your configuration related to binaries in the DXA docs, in particular for step 8. that the path to the binary extraction libraries (...
If you take a look in the source of the JPASearchBinaryContentDAO class, you'll see that error is thrown when there are no indexable file extensions configured. This happens if either the entire configuration is null, or if there is no "Indexer" node with a "DocExtensions" attribute. This is configured in your cd_storage_conf.xml, an example of which can be ...
I did the same activity in one of my project. You need to add the Enable Search Indexing TBB on your dynamic CT.
Also if you wants to have your own custom fields to add in SOLR.
Update the SearchIndexCustomFieldMap parameters of Enable Search Indexing
Add fields in SOLR managed-schema.xml file
The format of the map definition is as follows:
My understanding of SI4T is that the index data is appended as comment in the rendering phase to the rendered (Page) output, and on the CD side, this is (from a storage extension):
Sent to indexing to Solr (or some other system)
Stripped from the received output and persisted without it into the Content Data Store (Broker or filesystem).
Having this in ...
When you see these messages:
[11/25/14 16:41:02:649 CST] 0000005d SolrIndexDisp I org.si4t.solr.SolrIndexDispatcher getSolrServer Obtaining Http Solr server [http://aixpcibld01:9191/solr/collection1: http://aixpcibld01:9191/solr/collection1
[11/25/14 16:41:02:657 CST] 0000005d SolrIndexer I org.si4t.solr.SolrIndexer commit Clearing out registers.
In this ...
That is correct. Essentially, a Page Update is always done by the underlying JPA implementation as an JPASearchPageDAO.update, followed by a JPASearchPageDAO.create, so in essence both are doing the same things. This has got me stuck quite a while when I was building this as well.
If you want to change that behaviour because you have to do extra stuff while ...
It sounds like your question is about having SI4T index a whole category when it is published from the CMS. This is not possible, as only content from publishing pages and components is indexed (the storage extension works by extracting indexing directives from the rendered output of the page or dynamic component presentation - when you publish a category ...
There are a couple of things to check to see whether the setup has been configured properly.
Can you post the relevant parts of your cd_storage_conf.xml ? Have you for instance configured the SearchDAOBundle.xml file in the storage configuration?
Are all the relevant jar files placed in the lib directory of the deployer?
Can you post your logback ...
Given the lack of further information, I can only give generic troubeshooting hints.
1. Solr Configuration for indexing binaries
The most probably cause is that on the Solr side, you have not configured the data import handlers for indexing binaries properly. You need to ensure that all jar files which perform the imports are present in the lib directory ...
My suspicion is the the problem lies in ElasticSearchIndexer. Consider the following code:
private String publicationId = null;
private String indexname = null;
public void commit(String publicationId) throws IndexingException
System.out.println("Commit: "+ publicationId);
log.debug("ElasticsearchIndexer - commit called: " + ...
Actually you can achieve this via templating, no need for custom resolver. You can add additional dwt in your page template in which you will have RenderComponentPresentation() function for your metadata component with one or more defined templates. After that you can have your standard dwt for page that will generate your output.
With this approach you ...
In Sites 9 Regions are newly introduced to structure the content on a page in a better way. So the page is having a new structure now. The idea is while publishing the page we need to handle this region on the SI4T template. You can find the change in the pull request https://github.com/SI4T/SI4T/pull/21 . It will be merged in the next release I guess.