To answer the question on why you can't get results while searching on strings inside quotes or strings which have special characters in them like a : is to go down the Solr rabbit hole and find out why that actually is.
First of all, there are a couple of characters which always have to be escaped when performing a search query. In Solr these are: + - && || ! ( ) { } [ ] ^ " ~ * ? : \
So a search on:
((string.IsNullOrEmpty("FOO") && !(bar)) : 10+32 ? 43-1;
has to become:
\(\(string.IsNullOrEmpty\(\"FOO\"\) \&& \!\(bar\)\) \: 10\+32 \? 43\-1;
.. if you ever want to search for it.
Then there is the question on why you can't search on nav:navigation when you know you have a TBB which for example contains the following:
<style type="text/css" class="nav:navigation"> table{color: #333; text-align: left;</style>
The reason you will not be able to find this string in a template in Tridion, is because Tridion indexes the content of a TBB in a field called ContentText, which is defined in Tridion's Solr configuration schema.xml file, by using Solr's solr.WhitespaceTokenizerFactory, which in turn is defined as a Solr field type called 'text':
<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.ReversedWildcardFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
The above basically means that upon tokenizing input (aka indexing a search item), the string is split on whitespace and this means that only the values of the splitted string can actually be searched on. So in the above example, you would get a result back if you searched on:
class=\"nav\:navigation\">
.. simply because this string is actually which is the exact token on which can be searched. Also notice another caveat: the < and > characters are sometimes converted to their < and > equivalents, so it is also important to know when to enter the html entity variant if you want to search for the above string. I suspect that it depends whether you for instant search for converted markup inside XML or not.
This also explains why searching for *nav\:navigation* will yield a result, because if the wildcard token (*) is used as prefix and suffix, Solr will be able to match a complete token (in this case class="nav:navigation">), which happends to be surrounded by whitespace.
You can test all the above by going to your Solr instance on tridion:8983/tridion/admin. The other interesting bit you'll see here is that if you turn on highlighting in the Solr admin, you will see that Solr can only highlight the complete token and not substrings of that token.
However
It would be really nice to also be able to search for just "nav"and get back the above result. After a lot of fiddling around with Solr's different tokenizer classes, I came upon a simple solution: simply replace all non alpha numeric characters by whitespace prior to tokenizing content. In this test case it was sufficient, although in the real world additional analyzing and stemming of how to index Tridion content and template code properly is probably necessary. The way to do this is to change the "text" field type of Tridion's Solr configuration:
<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<charFilter
class="solr.PatternReplaceCharFilterFactory"
pattern="([^a-zA-Z0-9])"
replacement=" "
replace="all"/>
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.ReversedWildcardFilterFactory"/>
</analyzer>
<analyzer type="query">
<charFilter
class="solr.PatternReplaceCharFilterFactory"
pattern="([^a-zA-Z0-9])" replacement=" "
replace="all"/>
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
Running a solr.PatternReplaceCharFilterFactory prior to actually tokenizing content, whereby all non-alphanumeric characters are replaced by whitespace, will do the trick of having only alphanumeric tokens.
If the search host is restarted and the search index is reindexed, you will see that you will be able to just search for nav and get back the proper result. Note that this does not change what is stored in the index, but just what is tokenized. In this example, searching for class=\"nav\:, gives back:
<style type="text/css" <em>class</em>="<em>nav</em>:navigation">
table{color: #333; text-align:left;</style>
and searching for nav gives back:
<style type="text/css" class="<em>nav</em>:navigation">
table{color: #333; text-align:left;</style>
where the strings between the <em> tags are the actual tokens found and thus highlighted, and the token delimiters =," and : are not. And that is exactly what we would need if we were to search for meaningful code fragments in a proper way, so I'll probably always will change the field type configuration on my development machines (only) from now on. :)
