Data Loading

Migration SDK loads data during migration for purposes such as:

Finding content items to migrate.
Mapping references between content items (e.g. ownership, permissions, etc.) to valid destination items.
Converting Tableau Server schedules to Tableau Cloud schedules.

To efficiently handle migrations of large sites, the default data loading behavior loads all items, using the batch size for paging, and caches necessary information for future use. In most cases this "bulk loading" reduces total API calls compared to loading items individually, which minimizes API throttling and overall migration time.

For migration of large sites where only a few items are necessary, however, the time spent loading all items may exceed the time savings of reduced API calls. In this situation data loading can be configured through the plan builder.

If a content type with many items has already migrated but few are actually referenced, for example when user migration has already been completed, skipping the content type with pre-caching disabled is recommended.
In advanced scenarios where custom logic is needed, migration services can be registered to control data loading behavior.

Migration Content Loader

A migration content loader is responsible for finding the content items from the source site that should be considered for migration. Loaders produce batches of content items that are then mapped and filtered before migration. Items returned by the loader are added to the manifest and used when finding content references on the source site.

Custom migration content loaders can be used by overriding the IMigrationContentLoader migration service, either for all content types or a specific content type. See the Custom Migration Services topic for details on overriding migration services.

Python
C#

from typing import TypeVar
from tableau_migration import (
    empty_pager,
    MigrationContentLoaderBase
)

TContent = TypeVar("T")

class EmptyMigrationContentLoader(MigrationContentLoaderBase[TContent]):
    def get_migration_content_pager(self, page_size: int):
        return empty_pager(TContent)

public class EmptyMigrationContentLoader<TContent> : IMigrationContentLoader<TContent>
    where TContent : IContentReference
{
    public IPager<TContent> GetMigrationContentPager(int pageSize)
        => new MemoryPager<TContent>([], pageSize);
}

Content Reference Loading

When updating content references (e.g. a projects's owner ID), data from both source and destination sites are loaded as necessary. Content reference data found through the normal course of the migration are cached automatically for use in future content types, but when content items are referenced that were filtered out or otherwise excluded from migration, additional data load attempts are made by the cache before producing migration errors.

Custom content reference loading behavior can be used by overridding the IContentReferenceCacheLoadStrategyProvider migration service, either for all content types or a specific content type. Built-in providers are are available for the default bulk loading and lazy loading are available. See the Custom Migration Services topic for details on overriding migration services.

Python
C#

plan_builder.services.set(ContentReferenceCacheLoadStrategyProviderBase[IUser], LazyContentReferenceCacheLoadStrategyProvider[IUser])

planBuilder.Services.Set<IContentReferenceCacheLoadStrategyProvider<IUser>>(MigrationServiceFactoryContext ctx => 
{
	return ctx.Services.GetRequiredService<LazyContentReferenceCacheLoadStrategyProvider<IUser>>();
});