What's the fastest way to get all Components content using Core Service?

Question

I'm using Tridion 2011 SP1 and having issues getting all Components of a specific schema using the CoreService.

string schemaUri = "tcm:2-10-8"; // My Schema Uri
var filter = new UsingItemsFilterData { ItemTypes = new[] { ItemType.Component } };
XElement componentListElement = client.GetListXml(schemaUri, filter);                                                       
// This loop will take a LONG time to execute. I have about 4000 components of this type.
foreach (XElement component in componentListElement.Elements())
{
   var data = (ComponentData)client.Read(component.Attribute("ID").Value, 
                                                                  new ReadOptions());
   // do something with data.Content
}

Is there a more efficient way to do this?

Answer 1

Depending on what you're doing in your loop, and how busy your CM machine is while you're executing this, you could look into implementing your loop logic using Parallel.ForEach

Do be aware that this will possibly hammer your server's CPU and your database too.

UPDATE

Using GetList is unfortunately not an option with Tridion 2011. In Tridion 2013 this code does work:

UsingItemsFilterData f = new UsingItemsFilterData {ItemTypes = new[] {ItemType.Component}};
IdentifiableObjectData[] items = client.GetList("tcm:1043-7444-8", f);
foreach (var item in items)
{
    ComponentData component = (ComponentData) item;
    //Do something
}

UPDATE 2

Given that using Tridion 2013 is not an option here, then what I would try to do is to give the user the perception of speed - even if it's not that fast...

Not knowing exactly what you're trying to do - though it seems like it is linked to content validation from your comments - and assuming that you display results in some form of grid, I would try to use AJAX and CoreService async calls to update the list as the components get loaded and validated. This will add complexity to your code, but increase scalability without annoying your editors (it is usually fine if it takes longer to do something as long as you start showing results quicker).

Yet another option would be to validate your content on save, "mark" the components with a specific Application ID in AppData, then use this information to limit the number of components you need to load (you can get all components with a specific ApplicationID in one call).

Answer 2

A fast way I've found this is to pre-cache the data ahead of time. This works because the user does not have a requirement for the data to be real-time or up-to-the-minute. The report includes the Date/Time the data was cached, and gives them an option to re-request the data.

I ended up with a Console application run as a Scheduled task that dumps the components I'm interested in to a file the application has access to.

Answer 3

The best way to create custom tool to fetch the data or content of all component from Tridion_Cm Database with respect of Version,PublicationID and Schema Item ID.

might be it works in Your case if you need for reporting purpose.