When I publish a component that has "Add to Smart Target TBB" tridion reports its a success, but I noticed in the Fredhopper side the content is not indexed.
However when I remove text from the component to reduce its size it publishes correctly to Fredhopper, I noticed the size threshold is about 40 KB.
The attributes (fields) in Fredhopper have a maximum size which you probably exeeded. An attribute cannot exceed the max character length of 32K. Most likely you have hit this limit.
Logs
Check the Fredhopper logs on the indexer instance and the xmlloader log in particular. You can expect an error message like this:
2015-01-09 14:59:55,963 ERROR [XML-Parser-Thread] FredhopperHandler.java:354 10499 Rejecting faulty item `tcm_73-69473-16_tcm_73-68720-32'. || Administrator@2012r2:C:\fredhopper\data\installations\FAS tags/fas/fas-7.5.1.3/engine@109233
Causes
Having this long attributes could be caused by a lot of content in a Tridion content or metadata field, or a large rendered component presentation. Both Tridion fields and the output of the rendered component presentation are stored in Fredhopper as attributes which each cannot be longer than 32K characters.
If you are using DD4T the error can be caused by the verbose XML format for the published dynamic component presentation. The more fields, template metadata, embedded schemas and linked components you have, the longer the DD4T XML will be and at some point it can exeed the 32 Fredhopper attribute limit.
Solution
There is a SmartTarget hotfix ST_2014.1.0.88499 (you have to be logged in to https://www.sdltridionworld.com before downloading this) which both trims long fields and omits the component presentation. Just deploy the new smarttarget_core.jar from the hotfix to your Tridion Deployers.
Update: Updated Solution section: there is a hotfix for this issue which fixes both casuses.
