Configuration

The Migration SDK uses two sources of configuration

Basic Configuration that uses the Migration Plan. It contains configuration for a specific migration run.
Advanced Configuration. This configuration that is unlikely to change between migration runs.

Configuration Blocks

Basic Configuration

The bare minimum Migration SDK configuration is done using a Migration Plan. It defines the source, destination, and hooks executed during migration. The easiest way to generate a new Migration Plan is using MigrationPlanBuilder(IMigrationPlanBuilder implementation). Before you build a new plan, you need to:

Define Source.
Define Destination.
Define the Migration Type.
Customize with hooks (optional).

Important

Personal access tokens (PATs) are long-lived authentication tokens that allow you to sign in to the Tableau REST API without requiring hard-coded credentials or interactive sign-in. Best practices

Revoke and generate a new PAT every day to keep your server secure.
Access tokens should not be stored in plain text in application configuration files. Instead, use secure alternatives, such as encryption or a secrets management system.
If the source and destination sites are on the same server, use separate PATs.

Source

The method MigrationPlanBuilder.FromSourceTableauServer defines the source server by instantiating a new TableauSiteConnectionConfiguration with the following parameters:

serverUrl
siteContentUrl (optional)
accessTokenName
accessToken

Destination

The method MigrationPlanBuilder.ToDestinationTableauCloud defines the destination server by instantiating a new TableauSiteConnectionConfiguration with the following parameters:

podUrl
siteContentUrl: This is the site name on Tableau Cloud.
accessTokenName
accessToken

Migration Type

The method MigrationPlanBuilder.ForServerToCloud defines the migration type and load all default hooks for a Server to Cloud migration.

Add Hooks (optional)

The Plan Builder exposes the properties MigrationPlanBuilder.Hooks, MigrationPlanBuilder.Filters, MigrationPlanBuilder.Mappings, and MigrationPlanBuilder.Transformers. With these properties, you can customize your migration plan. See Custom Hooks article for more details.

Build

The method MigrationPlanBuilder.Build generates a Migration Plan ready to be used as an input to a migration process.

Advanced configuration

MigrationSdkOptions is the configuration class the Migration SDK uses internally to process a migration. It contains adjustable properties that change some engine behaviors. These properties are useful tools to troubleshoot and tune a migration process.

Note

Unless specified otherwise, all configuration options are dynamically applied during the migration.

Python
C#

Configuration values are set via environment variables. The : delimiter doesn't work with environment variable hierarchical keys on all platforms. For example, the : delimiter is not supported by Bash. The double underscore (__), which is supported on all platforms, automatically replaces any : delimiters in environment variables. All configuration environment variables start with MigrationSDK__.

ContentTypes

This is an array of MigrationSdkOptions.ContentTypesOptions. Each array object corresponds to settings for a single content type.

Important

The type values are case-insensitive. Duplicate type key values will result in an exception.

Python
C#

Python Environment Variables

MigrationSDK__ContentTypes__<array index>__<content type config key>__Type.
MigrationSDK__ContentTypes__<array index>__<content type config key>__BatchSize.

Example: To set the User BatchSize to 201 and Project BatchSize to 203, you would set environment variables as follows. Note the array indexes. They tie the setting values together in the Migration SDK.

# User BatchSize is 201
MigrationSDK__ContentTypes__0__Type = User
MigrationSDK__ContentTypes__0__BatchSize = 201

# Project BatchSize is 203
MigrationSDK__ContentTypes__1__Type = Project
MigrationSDK__ContentTypes__1__BatchSize = 203

In the following json example config file,

A BatchSize of 201 is applied to the content type User.
A BatchSize of 203 for Project.
A BatchSize of 200 for ServerExtractRefreshTask.

{
    "MigrationSdkOptions": {
        "contentType": [
        {
            "type":"User",
            "batchSize": 201            
        },        
        {
            "type":"Project",
            "batchSize": 203            
        },
        {
            "type":"ServerExtractRefreshTask",
            "batchSize": 200
        }
        ],
    }
}

The following table describes each setting. They should always be set per content type as described previously. If a setting below is not set for a content type, the Migration SDK falls back to the default value.

Key	Description	Default	Python Environment Variable
`ContentTypes.Type`	Determines which content type the settings apply to. Only supported content types will be considered and all others will be ignored. This key comes from the interface for the content type. For example, the key for IUser is 'User'. Content type values are case insensitive.	Not applicable.	`MigrationSDK__ContentTypes__<array-index>__<type-key>__Type`
`ContentTypes.BatchSize`	Defines the page size of each list request. See Tableau REST API Paginating Results for details.	`100`	`MigrationSDK__ContentTypes__<array-index>__<type-key>__BatchSize`
`ContentTypes.BatchPublishingEnabled`	Selects the mode to publish a given content type. Important: This option is available only for Users.	`false`	`MigrationSDK__ContentTypes__<array-index>__<type-key>__BatchPublishingEnabled`
`ContentTypes.IncludeExtractEnabled`	Determines whether to include extracts. Default: enabled. Important: This option is available only for Workbooks.	`false`	`MigrationSDK__ContentTypes__<array-index>__<type-key>__IncludeExtractEnabled`

MigrationParallelism

This setting defines the number of parallel tasks migrating the same type of content simultaneously. You can tune the Migration SDK processing time with this configuration.

Default: 10 Python Environment Variable: MigrationSDK__MigrationParallelism

Warning

There are concurrency limits in REST APIs on Tableau Cloud. The current default configuration is the balance between performance without blocking too many resources to the migration process.

File

This section contains options related to file storage.

Key	Description	Default	Python Environment Variable
`Files.DisableFileEncryption`	Defines whether to encrypt temporary files downloaded during the migration. This applies to file-based content types, such as Workbooks and Data Sources.	`false`	`MigrationSDK__Files__DisableFileEncryption`
`Files.RootPath`	Defines the location to store temporary files.	`The default temporary path for the OS.`	`MigrationSDK__Files__RootPath`

Network

This configuration section contains network-related options.

Important

NetworkOptions.UserAgentComment is not dynamically applied. It takes effect when you restart your application.

Key	Description	Default	Python Environment Variable
`Network.FileChunkSizeKB`	The chunk size (KB) for publishing large files. This applies to file-based content types like Workbooks and Data Sources. See the Tableau REST API Publishing Resources documentation for more details.	`65536`	`MigrationSDK__Network__FileChunkSizeKB`
`Network.HeadersLoggingEnabled`	Enables logging of HTTP request headers.	`false`	`MigrationSDK__Network__HeadersLoggingEnabled`
`Network.ContentLoggingEnabled`	Enables logging of HTTP request content.	`false`	`MigrationSDK__Network__ContentLoggingEnabled`
`Network.BinaryContentLoggingEnabled`	Enables logging of binary content in HTTP requests.	`false`	`MigrationSDK__Network__BinaryContentLoggingEnabled`
`Network.ExceptionsLoggingEnabled`	Enables logging of HTTP request exceptions.	`false`	`MigrationSDK__Network__ExceptionsLoggingEnabled`
`Network.UserAgentComment`	Defines a comment to append to the User-Agent header in all HTTP requests. This property is only used to assist in server-side debugging and it not typically set.		`MigrationSDK__Network__UserAgentComment`

Resilience

The Resilience sub-section deals with the resilience and transient-fault layer. See Microsoft.Extensions.Http.Resilience for more details.

Key	Description	Default	Python Environment Variable
`Network.Resilience.RetryEnabled`	Defines whether to retry failed requests.	`true`	`MigrationSDK__Network__Resilience__RetryEnabled`
`Network.Resilience.RetryIntervals`	Defines the number of retries and interval between retries.	[ `500` ms, `500` ms, `500` ms, `1` s, `2` s ]	Not supported
`Network.Resilience.RetryOverrideResponseCodes`	Overrides the default error status codes for retries.		Not supported
`Network.Resilience.ConcurrentRequestsLimitEnabled`	Defines whether to limit concurrent requests.	`false`	`MigrationSDK__Network__Resilience__ConcurrentRequestsLimitEnabled`
`Network.Resilience.MaxConcurrentRequests`	Defines the maximum quantity of concurrent API requests. This is based on the number of logical processors (or `processor count`).	(`processor count`)/`2`	`MigrationSDK__Network__Resilience__MaxConcurrentRequests`
`Network.Resilience.ConcurrentWaitingRequestsOnQueue`	Defines the quantity of concurrent API requests waiting on queue. This is also based on `processor count`.	(`processor count`)/`4`	`MigrationSDK__Network__Resilience__ConcurrentWaitingRequestsOnQueue`
`Network.Resilience.ClientThrottleEnabled`	Defines whether to limit requests to a given endpoint.	`false`	`MigrationSDK__Network__Resilience__ClientThrottleEnabled`
`Network.Resilience.MaxReadRequests`	Limits the amount of GET requests for the Client Throttle.	`40000`	`MigrationSDK__Network__Resilience__MaxReadRequests`
`Network.Resilience.MaxReadRequestsInterval`	Defines the interval for the Read Request Throttle.	`1` h	`MigrationSDK__Network__Resilience__MaxReadRequestsInterval`
`Network.Resilience.MaxPublishRequests`	Defines the maximum quantity of non-GET requests on the client side.	`5500`	`MigrationSDK__Network__Resilience__MaxPublishRequests`
`Network.Resilience.MaxPublishRequestsInterval`	Defines the interval for the limit of non-GET requests on the client side.	`1` day	`MigrationSDK__Network__Resilience__MaxPublishRequestsInterval`
`Network.Resilience.ServerThrottleEnabled`	Defines whether to retry requests throttled on the server.	`true`	`MigrationSDK__Network__Resilience__ServerThrottleEnabled`
`Network.Resilience.ServerThrottleLimitRetries`	Defines whether there is a limit of retries for a throttled request.	`false`	`MigrationSDK__Network__Resilience__ServerThrottleLimitRetries`
`Network.Resilience.ServerThrottleRetryIntervals`	Defines the interval between each retry for throttled requests without the 'Retry-After' header.	[ `1` s, `3` s, `10` s, `30` s, `1` m ]	Not supported
`Network.Resilience.PerRequestTimeout`	Defines the maximum duration of non-FileTransfer requests.	`30` m	`MigrationSDK__Network__Resilience__PerRequestTimeout`
`Network.Resilience.PerFileTransferRequestTimeout`	Defines the maximum duration of FileTransfer requests.	`12` h	`MigrationSDK__Network__Resilience__PerFileTransferRequestTimeout`

DefaultPermissionsContentType

Key	Description	Default	Python Environment Variable
`DefaultPermissionsContentTypes.UrlSegments`	List of types of default permissions for a given project. See Query Default Permissions for details. Important: This configuration is not dynamically applied. It takes effect when you restart your application.	Listed in `DefaultPermissionsContentTypeUrlSegments`	Not Supported

Job

The Migration SDK uses two methods to publish the content to a destination server:

'Bulk process': A single REST API call for multiple items.
'Individual process': One REST API call per item.

This configuration only applies to the 'Bulk process'. Each batch publish REST API call returns a Job ID (see the Tableau REST API Query Job for details). The SDK uses this ID to determine job status. The following table describes the related settings.

Key	Description	Default	Python Environment Variable
`Jobs.JobPollRate`	Defines the interval to wait to recheck processing status for bulk processes.	`3` s	`MigrationSDK__Jobs__JobPollRate`
`Jobs.JobTimeout`	Defines the maximum interval to wait for a job to complete for bulk processes.	`30` m	`MigrationSDK__Jobs__JobTimeout`