13

I'm using Tridion 2011 SP1 and having issues getting all Components of a specific schema using the CoreService.

string schemaUri = "tcm:2-10-8"; // My Schema Uri
var filter = new UsingItemsFilterData { ItemTypes = new[] { ItemType.Component } };
XElement componentListElement = client.GetListXml(schemaUri, filter);                                                       
// This loop will take a LONG time to execute. I have about 4000 components of this type.
foreach (XElement component in componentListElement.Elements())
{
   var data = (ComponentData)client.Read(component.Attribute("ID").Value, 
                                                                  new ReadOptions());
   // do something with data.Content
}

Is there a more efficient way to do this?

16
  • 2
    I can read 4000 fairly complex records with multiple associations from Entity Framework in under 30 seconds. I consider anything above the default ScriptTimeout for ASP.Net to be too slow.
    – JJS
    May 7, 2013 at 16:10
  • 1
    Entity framework would be getting all 4000 results in one database round trip (depending on approach). Your code above is doing 4001 database hits so will be significantly slower. May 7, 2013 at 16:43
  • 2
    The defaults for ASP.Net timeouts are presumably based on what would be reasonable for a public-facing web page. I wouldn't think the same criteria should apply to a batch job like this. But what's your use case? Mostly if you're copying 4000 components it's a one-off. I'd only bother optimising it if you're going to be doing it a lot. May 7, 2013 at 19:02
  • 1
    Have you tried using GetList? (note, it's not implemented everywhere in CoreService, but where it works it returns IEnumerable<IdentifiableObject>)
    – Nuno Linhares
    May 7, 2013 at 19:26
  • 1
    @DominicCronin - my use case is a content-editor facing report that shows which Campaigns have a duplicate value like sourceCode or vanityURL. It's not a really batch-job.
    – JJS
    May 7, 2013 at 19:54

3 Answers 3

8

Depending on what you're doing in your loop, and how busy your CM machine is while you're executing this, you could look into implementing your loop logic using Parallel.ForEach

Do be aware that this will possibly hammer your server's CPU and your database too.

UPDATE

Using GetList is unfortunately not an option with Tridion 2011. In Tridion 2013 this code does work:

UsingItemsFilterData f = new UsingItemsFilterData {ItemTypes = new[] {ItemType.Component}};
IdentifiableObjectData[] items = client.GetList("tcm:1043-7444-8", f);
foreach (var item in items)
{
    ComponentData component = (ComponentData) item;
    //Do something
}

UPDATE 2

Given that using Tridion 2013 is not an option here, then what I would try to do is to give the user the perception of speed - even if it's not that fast...

Not knowing exactly what you're trying to do - though it seems like it is linked to content validation from your comments - and assuming that you display results in some form of grid, I would try to use AJAX and CoreService async calls to update the list as the components get loaded and validated. This will add complexity to your code, but increase scalability without annoying your editors (it is usually fine if it takes longer to do something as long as you start showing results quicker).

Yet another option would be to validate your content on save, "mark" the components with a specific Application ID in AppData, then use this information to limit the number of components you need to load (you can get all components with a specific ApplicationID in one call).

6
  • I've run into this same problem. My script copies items from one publication to another and have found it takes around 8 hours to run. I did try using Parallel.ForEach assuming this would make it a lot quicker but when trying to build a Structure Group structure it failed due to the order of which items were being created. Will be interesting to see what approach people will recommend. May 7, 2013 at 15:44
  • 1
    As I mentioned, it depends on what you're doing in the loop - not all code can be parallelized. If the action you're doing is "ACID" then you should be fine with Parallel ForEach
    – Nuno Linhares
    May 7, 2013 at 15:51
  • @IbrarHussain - an approach I've used before is to loop through the structure and build your folders first. (mind the hierarchy), then, loop through a second time to create/copy/move your items.
    – JJS
    May 7, 2013 at 19:53
  • @NunoLinhares - while this is a valid answer for 2013, I can't mark it as correct because I need a solution for 2011.
    – JJS
    May 7, 2013 at 20:20
  • Would looping through the structure twice not make things even slower? There are approximately 1000 SGs. I basically have one publication from where i get all my data i.e. SGs, SG CLs, CLs RTF fields for CLs and lastly pages. All these items are recreated in a different publication which is why my console app takes 8 hours :-) May 8, 2013 at 6:30
5

A fast way I've found this is to pre-cache the data ahead of time. This works because the user does not have a requirement for the data to be real-time or up-to-the-minute. The report includes the Date/Time the data was cached, and gives them an option to re-request the data.

I ended up with a Console application run as a Scheduled task that dumps the components I'm interested in to a file the application has access to.

0

The best way to create custom tool to fetch the data or content of all component from Tridion_Cm Database with respect of Version,PublicationID and Schema Item ID.

might be it works in Your case if you need for reporting purpose.

3
  • Depending on what @JJS wants to do with the list of Components, this may not be an option. For example, if he then wants to update the contents or metadata on each one, this should not be done through a database query. Also, this approach is not supported by SDL, may invalidate your support and may not work if the database structure changes in a future version of the product. I agree that it would be fast though! Oct 1, 2018 at 11:17
  • Agreed Jonathan.
    – JAGAN
    Oct 1, 2018 at 12:22
  • Good solution @JAGAN, but accessing the database directly was forbidden in the environment. If you have the relevant SQL for this, you might consider offering it to help someone out.
    – JJS
    Oct 1, 2018 at 15:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.