Timeline for Possible threading problem in SI4T
Current License: CC BY-SA 3.0
16 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 15, 2016 at 14:17 | comment | added | Neil | No - I'm not sure I can as it was part of clients solution. | |
| Nov 30, 2016 at 20:54 | comment | added | Nuno Linhares♦ | @Neil - did you ever share this? Got someone interested in a similar solution | |
| Mar 10, 2015 at 11:46 | vote | accept | Neil | ||
| Mar 9, 2015 at 9:50 | comment | added | Raimond | Ok. Keep in mind I'm releasing a new version soon, which solves an issue with having multiple DAOFactory types. The current version won't break anything, but a bit more robust implementation will be in place. If you need this fast, check the code here: github.com/SI4T/SI4T/tree/feature/bugfixing/storage-extensions/… | |
| Mar 9, 2015 at 9:47 | answer | added | Raimond | timeline score: 4 | |
| Mar 9, 2015 at 8:55 | comment | added | Neil | Raimond, updated the storage conf example above (I removed the long Mapping string for brevity its only used when creating the index). This is a DD4T solution, so we use one JPASearchDAOFactory for pages and DCPs, one FSSearchDAOFactory for TXT files (we wanted this file stored in a specific place) and one FSSearchDAOFactory for binaries. | |
| Mar 9, 2015 at 8:50 | history | edited | Neil | CC BY-SA 3.0 |
added 914 characters in body
|
| Mar 9, 2015 at 7:38 | comment | added | Raimond | That might be. Do you use multiple DAOFactories for multiple item types as well ? | |
| Mar 8, 2015 at 21:11 | comment | added | Neil | "The config parts"? I chose to calculate index name in code rather than the Solr way of using config - could that be a root cause? | |
| Mar 8, 2015 at 19:35 | comment | added | Raimond | That's not possible in theory, but may be a bug if it is. Even though it is a singleton, I've built in guarantees that only one thread takes one transaction. Having said that, if you have to DAOFactories in use in the same transaction, you might need a bugfix I did a while back. But, even then it can never be that a transaction puts stuff in the wrong publication id because of the multi threadedness. I suspect it's rather the config parts which have an issue. | |
| Mar 8, 2015 at 14:34 | history | edited | Neil | CC BY-SA 3.0 |
added 2113 characters in body
|
| Mar 8, 2015 at 9:22 | comment | added | Neil | @Raimond - I'll update my answer with the info you're looking for. But its a good start to know each individual transaction only holds a single publications items. But the SearchIndexProcessor is a singleton - the Deployer is multi-threaded - is it possible different threads could call the triggerIndexing method therefore causing a commit cross-transactions? | |
| Mar 7, 2015 at 22:06 | comment | added | Raimond | Good, I wouldn't expect that. :) Then I suspect it's the way index urls are mapped to publication Ids, but I can't say much about it without knowing the setup. | |
| Mar 7, 2015 at 20:00 | comment | added | Nuno Linhares♦ | There are definitely no transactions with items from multiple publications. I just denied an ER asking for that ;) | |
| Mar 7, 2015 at 17:58 | comment | added | Raimond | I'm not aware of threading issues in SI4T, but you seem to have quite a complex setup. Figuring this out, would mean for me to know your storage configuration, access to example transaction packages and logging. Could you provide those? Everything the deployer does (and SI4T) happens on the transaction Id and never the publication id in the sense of changing that publication Id. It may be that one transaction has pages from multiple publications, but the standard deployer should take that into account. If not-then it's a definite CD bug or a bug in the way the index>pub mapping happens in SI4T | |
| Mar 7, 2015 at 13:28 | history | asked | Neil | CC BY-SA 3.0 |