5

I'm currently facing a problem that I cannot properly identify, so I will give the lengthy version in the hope that I could have missed some details.

I'm performing a content migration from Tridion R5.2 to Tridion 2011 with a 3rd party tool. To simplify, it involves scanning every items in the old system and creating them in the new one.

I'm constrained in the tooling that I can use, so I have to interact with the Business Connector (more precisely a webservice, located at http://<url>/Library/Commands/WebGUIResponder.asp in R5 and http://<url>/BCListener/BusinessConnectorService.asmx in 2011, to which I send SOAP requests).

The tool I use has 3 runtime options when perform the scan: commandline mode, which sends a lot of requests per seconds (by running 10 scripts in parallel), debug mode, which runs one script only, and design mode, in which I can execute the script line by line.

I'm not going to enter in too much details about how I found this problem, but here is the sequence of actions/results that I cannot explain:

  • I ran an initial scan in commandline.
  • I realized that ~50000 items are missing from my initial scan.
  • All those components are in somePublication/someFolders/folder, which itself contains ~2500 subfolders, each containing ~20 items.
  • I thought the script had a glitch (it can happen), so I ran in commandline a scan of folder only... but still none of those items were found
  • I decide to run the script in debug mode: the ~2500 subfolders are suddenly found...
  • I run in command line mode the scan of those 2500 subfolders, and finally, all 50000 items are found... Except 700.
  • I decide to modify the script and ask the content of those 700 tcmids only, in debug mode
  • It runs perfectly, except for ~50 of them, and the returned error is (by memory) "No component with this Id exists"
  • I inspect one of the IDs in design mode, and sure enough the content is returned.
  • So I ran some more scans in debug mode, on the tcmids for which I didn't get the content after each run, with this (approximate) number of errors for every consecutive run: 25, 10, 5, 3, 3, 2, 2, 0 (i.e. I got the content of 25 items on the first run, then 15, 5, 2, 0, 1, 0, 0, 2).

Basically, it seems that when I ask for the content of an item, tridion will either give it to me, either say that the item does not exist. Completely randomly.

What I'm looking for here is some hints as why this behavior.

  • 1
    Single script returning smaller and smaller sets of errors--maybe look to connectivity and/or timeout issues. For missing sets of content, I'd double check it's the correct CMS instance, permissions if not already running as an admin, or even if the components need synching. Other than that, maybe the tcm ids are wrong? – Alvin Reyes Mar 16 '13 at 20:35
  • +1 for the synching option. Your last run returns 0 error. Does an item that fails the first time (either in the GUI or in the tool) after inspecting it in the GUI (opening and closing it) fail a second time? If not synching is a candidate... – Albert Romkes Mar 16 '13 at 21:34
  • 1
    -1 for making the 3rd party tool a mystery. What is it? And what on earth is the reference to quantum computing? Is that the tool? – Dominic Cronin Mar 16 '13 at 23:02
  • @AlvinReyes: I've been unclear about the small number of errors: I ran the script to get the content of the items for which I had none after each subsequent run. And in the end, the fact that I had 0 errors means that all the tcmids were correct, since I could get the data for every of them. Also, the script is written to handle connectivity and timeout issues (basically I loop infinitely if I got a connection issue). I don't think it's a permission issue, since I could scan the rest of the 3 millions items without problems. I'd like to know more about this syncing though. – m4573r Mar 16 '13 at 23:19
  • 1
    @DominicCronin: How is the tool relevant?? I send soap calls to a tridion webserver. Whether I generate the SOAP requests with bash script, a java program or whatnot doesn't matter the least, and certainly doesn't deserve a -1... As for quantum computing, it was a humorous reference joke to the fact that laymen associate quantum computing with the uncertainty of the results. In my case, I ran 10 times the same script on the same tcmids, and I got 10 times a different result. – m4573r Mar 16 '13 at 23:24
3

Obviously something's going on - other than quantum computing - with the responder. I find it interesting that this tool uses the WebGUIResponder.asp (internal, used by the GUI) instead of the Business Connector webservice (which uses the same API and is a public interface available since god-knows-when, it was there when I started working with Tridion with 5.1 SP2), and then uses the Business Connector on Tridion 2011 - it could have used both in both releases with minimal changes since it's the same API.

Not saying that this is the cause of your issues, but interesting fact anyway.

My take on it...

  • Is the use of a migration tool really required? Can't you achieve the same with a database upgrade followed by clean-up?
  • Given that you can't change much about the tool itself, I'd probably go down the route of determining what's at fault here - Business Connector vs WebGUIResponder.asp - basically write a small script that does the same operation using one interface vs the other, and compare results.
  • Jeez man, 5.2? Really? That's been deprecated 4 years ago. (nothing you can do about it, but I had to say it)

In short, my recommendation would be to upgrade the database and clean up afterwards. If you're doing additional content clean up as part of the migration, then see if you can decouple content extraction & loading from the tool and do it yourself with WCF/CoreService on Tridion 2011 (Upgrade DB, hook it to a Tridion 2011 instance, extract content using CoreService, perform your migration updates, load it to a new Tridion 2011 instance again using CoreService).

  • +1 for the upgrade db and clean-up approach. As for 5.2, m4573r is upgrading /from/ 5.2 to a better place :-) – Dominic Cronin Mar 17 '13 at 8:38
  • Fisrt point: The tool is used on complement of a database updrage. Basically, since the database upgrade/cleanup is very slow, the tool is supposed to quickly pick the changes on the live R5 environment and migrate then to 2011, and it's supposed to minimize the content freeze. Second and third point: Yes, from 5.2, and would you believe it it's the second migration of this type I do this month. The first one went ok. As a result, the /BCListener/BusinessConnectorService.asmx doesn't exist in the source system. – m4573r Mar 17 '13 at 11:06
  • As for your recommendation of decoupling the extration and the import, it's also a road I'm am planning to take, but the point of my question was to understand why I got those inconsistent reads. – m4573r Mar 17 '13 at 11:07
  • The answer is "I have no clue". Sorry – Nuno Linhares Mar 17 '13 at 12:12
1
  1. Could you be hitting time-out errors of some sort? The fact that you see a difference between the bulk mode and single item mode would be an indicator of this. (When doing bulk imports, it's often useful to refresh the database statistics at intervals during the import.)
  2. Are you seeing any errors in the logging output from Tridion. Please check the relevant Windows event logs
  3. The WebUIResponder.asp endpoint is not a public API. You should probably be using the Business Connector instead, however, in practice, the responder is likely to work as well as (or better than) the Business connector.
  4. When you run your tool in its various modes, are you using the same user account in Tridion. If an item is not properly checked in it may be invisible to other users.
  • 1) My assumption was that if this happens, I would get a proper error message from the webserver, not 'this component doesn't exist'. 2) I checked the logs and nothing appears here (well, there are errors, but not related to this activity). 3) Since you and Nuno are suggesting that, I'll look into it again, but as I told him, I'm pretty sure /BCListener/BusinessConnectorService.asmx doesn't exists in R5. 4) Yes, I was given one single account to access tridion. – m4573r Mar 17 '13 at 11:11
  • The URL to the business connector may have changed, but it certainly did exist in R5. – Nuno Linhares Mar 17 '13 at 12:14
  • 1
    The business connector would only be there if it was installed. – Dominic Cronin Mar 17 '13 at 13:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.