Introduction

Welcome to the developer documentation for the Tableau Migration SDK. The Migration SDK is primarily written in C# using the .NET Framework. It includes a Python API to enable interoperability with Python.

Supported languages

You can develop your migration application using one of the supported languages

Python
C# using the .NET Framework

Versioning

The Migration SDK uses semantic versioning.

Versioning

Prerequisites

To develop your application using the Migration SDK, you should

Understand framework and language concepts for one of the languages the SDK supports.
Be able to design and write applications in one of the supported languages.
Understand how to import and use third party packages
Understand logging and general troubleshooting of applications.

Quick Start

Install a .NET Runtime.
Install the Migration SDK

Python
C#

Install using PIP

PIP CLI: pip install tableau_migration

Use the sample code in Example startup code to get started.
Use the Resource sections to further customize your application.

Example startup code

The following code samples are for writing a simple migration app using the Migration SDK. For details on configuring and customizing the Migration SDK to your specific needs, see the other articles and Code Samples.

Python
C#

# This application performs a basic migration using the Tableau Migration SDK.
# By default all supported content will be migrated, but can be modified to your specific needs.
# The application assumes you have already installed the Tableau Migration SDK Python package.

from dotenv import load_dotenv

load_dotenv()

import configparser          # configuration parser
import os                    # environment variables
import tableau_migration     # Tableau Migration SDK
from print_result import print_result

from threading import Thread # threading

from tableau_migration import (
    MigrationManifestSerializer,
    MigrationManifest
)

serializer = MigrationManifestSerializer()

def migrate():
    """Performs a migration using Tableau Migration SDK."""
    
    # Get the absolute path of the current file
    current_file_path = os.path.abspath(__file__)
    manifest_path = os.path.join(os.path.dirname(current_file_path), 'manifest.json')

    plan_builder = tableau_migration.MigrationPlanBuilder()
    migration = tableau_migration.Migrator()

    config = configparser.ConfigParser()
    config.read('config.ini')

    # Build the plan.
    plan_builder = plan_builder \
                    .from_source_tableau_server(
                        server_url = config['SOURCE']['URL'], 
                        site_content_url = config['SOURCE']['SITE_CONTENT_URL'], 
                        access_token_name = config['SOURCE']['ACCESS_TOKEN_NAME'], 
                        access_token = os.environ.get('TABLEAU_MIGRATION_SOURCE_TOKEN', config['SOURCE']['ACCESS_TOKEN']),
                        create_api_simulator = os.environ.get('TABLEAU_MIGRATION_SOURCE_SIMULATION', 'False') == 'True') \
                    .to_destination_tableau_cloud(
                        pod_url = config['DESTINATION']['URL'], 
                        site_content_url = config['DESTINATION']['SITE_CONTENT_URL'], 
                        access_token_name = config['DESTINATION']['ACCESS_TOKEN_NAME'], 
                        access_token = os.environ.get('TABLEAU_MIGRATION_DESTINATION_TOKEN', config['DESTINATION']['ACCESS_TOKEN']),
                        create_api_simulator = os.environ.get('TABLEAU_MIGRATION_DESTINATION_SIMULATION', 'False') == 'True') \
                    .for_server_to_cloud() \
                    .with_tableau_id_authentication_type() \
                    .with_tableau_cloud_usernames(config['USERS']['EMAIL_DOMAIN'])                    

    # TODO: add filters, mappings, transformers, etc. here.


    # Load the previous manifest file if it exists.
    prev_manifest = load_manifest(f'{manifest_path}')

    # Validate the migration plan.
    validation_result = plan_builder.validate()

    # TODO: Handle errors if the validation fails here.

    plan = plan_builder.build()

    # Run the migration.
    results = migration.execute(plan, prev_manifest)
    
    # Save the manifest file.
    serializer.save(results.manifest, f'{manifest_path}')

    # TODO: Handle results here.
    print_result(results)

    print("All done.")

def load_manifest(manifest_path: str) -> MigrationManifest | None:
        """Loads a manifest if requested."""
        manifest = serializer.load(manifest_path)
    
        if manifest is not None:
            while True:
                answer = input(f'Existing Manifest found at {manifest_path}. Should it be used? [Y/n] ').upper()

                if answer == 'N':
                    return None
                elif answer == 'Y' or answer == '':
                    return manifest
                
        return None
    

if __name__ == '__main__':
    
    # Create a thread that will run the migration and start it.
    migration_thread = Thread(target = migrate)
    migration_thread.start()
    done = False

    # Create a busy-wait loop to continue checking if Ctrl+C was pressed to cancel the migration.
    while not done:
        try:
            migration_thread.join(1)
            done = True
        except KeyboardInterrupt:
            # Ctrl+C was caught, request migration to cancel. 
            print("Caught Ctrl+C, shutting down...")
            
            # This will cause the Migration SDK to cleanup and finish,
            # which will cause the thread to finish.
            tableau_migration.cancellation_token_source.Cancel()
            
            # Wait for the migration thread to finish and then quit the application.
            migration_thread.join()
            done = True

Important

The values below should not be quoted. So no ' or ".

[SOURCE]
URL = http://server
SITE_CONTENT_URL = 
ACCESS_TOKEN_NAME = MyServerTokenName
ACCESS_TOKEN = 

[DESTINATION]
URL = https://pod.online.tableau.com
SITE_CONTENT_URL = mycloudsite
ACCESS_TOKEN_NAME = MyCloudTokenName
ACCESS_TOKEN = 

[USERS]
EMAIL_DOMAIN = mycompany.com

python-dotenv==1.0.1

namespace Csharp.ExampleApplication
{
    public static class Program
    {
        public static async Task Main(string[] args)
        {
            // Set the DOTNET_ENVIRONMENT environment variable to the name of the environment.
            // This loads the appsettings.<DOTNET_ENVIRONMENT>.json config file.
            // If no DOTNET_ENVIRONMENT is set, appsettings.json will be used
            using var host = Host.CreateDefaultBuilder(args)
                .ConfigureServices((ctx, services) =>
                {
                    services
                        .Configure<MyMigrationApplicationOptions>(ctx.Configuration)
                        .Configure<EmailDomainMappingOptions>(ctx.Configuration.GetSection("tableau:emailDomainMapping"))
                        .Configure<UnlicensedUsersMappingOptions>(ctx.Configuration.GetSection("tableau:unlicensedUsersMapping"))
                        .AddTableauMigrationSdk(ctx.Configuration.GetSection("tableau:migrationSdk"))
                        .AddCustomizations()
                        .AddHostedService<MyMigrationApplication>();
                })
                .Build();

            await host.RunAsync();
        }

        /// <summary>
        /// Registers services required for using the Tableau Migration SDK customizations.
        /// </summary>
        /// <param name="services">The service collection to register services with.</param>        
        /// <returns>The same service collection as the <paramref name="services"/> parameter.</returns>
        public static IServiceCollection AddCustomizations(this IServiceCollection services)
        {
            services.AddScoped<CustomContext>();

            services.AddScoped<SetMigrationContextHook>();

            services.AddScoped<EmailDomainMapping>();

            services.AddScoped<UnlicensedUsersMapping>();

            services.AddScoped<ProjectRenameMapping>();

            services.AddScoped<ChangeProjectMapping<IWorkbook>>();
            services.AddScoped<ChangeProjectMapping<IDataSource>>();

            services.AddScoped<DefaultProjectsFilter>();

            services.AddScoped<UnlicensedUsersFilter>();
            
            services.AddScoped<SharedCustomViewFilter>();

            services.AddScoped(typeof(UpdatePermissionsHook<,>));

            services.AddScoped(typeof(BulkLoggingHook<>));

            services.AddScoped<MigratedTagTransformer<IPublishableDataSource>>();
            services.AddScoped<MigratedTagTransformer<IPublishableWorkbook>>();

            services.AddScoped<EncryptExtractsTransformer<IPublishableDataSource>>();
            services.AddScoped<EncryptExtractsTransformer<IPublishableWorkbook>>();

            services.AddScoped(typeof(SimpleScheduleStartAtTransformer<>));
            
            services.AddScoped<CustomViewExcludeDefaultUserTransformer>();

            services.AddScoped<LogMigrationActionsHook>();

            services.AddScoped(typeof(LogMigrationBatchesHook<>));

            return services;
        }
    }
}


namespace Csharp.ExampleApplication
{
    internal sealed class MyMigrationApplication : IHostedService
    {
        private readonly Stopwatch _timer;
        private readonly IHostApplicationLifetime _appLifetime;
        private IMigrationPlanBuilder _planBuilder;
        private readonly IMigrator _migrator;
        private readonly MyMigrationApplicationOptions _options;
        private readonly ILogger<MyMigrationApplication> _logger;
        private readonly MigrationManifestSerializer _manifestSerializer;

        public MyMigrationApplication(
            IHostApplicationLifetime appLifetime,
            IMigrationPlanBuilder planBuilder,
            IMigrator migrator,
            IOptions<MyMigrationApplicationOptions> options,
            ILogger<MyMigrationApplication> logger,
            MigrationManifestSerializer manifestSerializer)
        {
            _timer = new Stopwatch();

            _appLifetime = appLifetime;

            // You can choose to assign an instance of the ServerToCloudMigrationPlanBuilder to help you 
            // add your own filters, mappings, transformers or hooks.
            // Refer to the Articles section of this documentation for more details.
            _planBuilder = planBuilder;
            _migrator = migrator;
            _options = options.Value;
            _logger = logger;
            _manifestSerializer = manifestSerializer;
        }

        public async Task StartAsync(CancellationToken cancel)
        {
            var executablePath = Assembly.GetExecutingAssembly().Location;
            var currentFolder = Path.GetDirectoryName(executablePath);
            if (currentFolder is null)
            {
                throw new Exception("Could not get the current folder path.");
            }
            var manifestPath = $"{currentFolder}/manifest.json";

            var startTime = DateTime.UtcNow;
            _timer.Start();

            // Use the methods on your plan builder to add configuration and make customizations.
            _planBuilder = _planBuilder
                .FromSourceTableauServer(_options.Source.ServerUrl, _options.Source.SiteContentUrl, _options.Source.AccessTokenName, Environment.GetEnvironmentVariable("TABLEAU_MIGRATION_SOURCE_TOKEN") ?? string.Empty)
                .ToDestinationTableauCloud(_options.Destination.ServerUrl, _options.Destination.SiteContentUrl, _options.Destination.AccessTokenName, Environment.GetEnvironmentVariable("TABLEAU_MIGRATION_DESTINATION_TOKEN") ?? string.Empty)
                .ForServerToCloud()
                .WithTableauIdAuthenticationType()
                // You can add authentication type mappings here            
                .WithTableauCloudUsernames<EmailDomainMapping>();

            var validationResult = _planBuilder.Validate();

            if (!validationResult.Success)
            {
                _logger.LogError("Migration plan validation failed. {Errors}", validationResult.Errors);
                Console.WriteLine("Press any key to exit");
                Console.ReadKey();
                _appLifetime.StopApplication();
            }

            // Add mappings
            _planBuilder.Mappings.Add<UnlicensedUsersMapping, IUser>();

            _planBuilder.Mappings.Add<ProjectRenameMapping, IProject>();

            _planBuilder.Mappings.Add<ChangeProjectMapping<IDataSource>, IDataSource>();
            _planBuilder.Mappings.Add<ChangeProjectMapping<IWorkbook>, IWorkbook>();

            // Add filters
            _planBuilder.Filters.Add<DefaultProjectsFilter, IProject>();

            _planBuilder.Filters.Add<UnlicensedUsersFilter, IUser>();
            
            _planBuilder.Filters.Add<SharedCustomViewFilter, ICustomView>();

            // Add post-publish hooks
            _planBuilder.Hooks.Add<UpdatePermissionsHook<IPublishableDataSource, IDataSourceDetails>>();
            _planBuilder.Hooks.Add<UpdatePermissionsHook<IPublishableWorkbook, IWorkbookDetails>>();

            _planBuilder.Hooks.Add<BulkLoggingHook<IUser>>();

            // Add transformers
            _planBuilder.Transformers.Add<MigratedTagTransformer<IPublishableDataSource>, IPublishableDataSource>();
            _planBuilder.Transformers.Add<MigratedTagTransformer<IPublishableWorkbook>, IPublishableWorkbook>();

            _planBuilder.Transformers.Add<EncryptExtractsTransformer<IPublishableDataSource>, IPublishableDataSource>();
            _planBuilder.Transformers.Add<EncryptExtractsTransformer<IPublishableWorkbook>, IPublishableWorkbook>();

            _planBuilder.Transformers.Add<SimpleScheduleStartAtTransformer<ICloudExtractRefreshTask>, ICloudExtractRefreshTask>();
            
            _planBuilder.Transformers.Add<CustomViewExcludeDefaultUserTransformer, IPublishableCustomView>();

            // Add initialize migration hooks
            _planBuilder.Hooks.Add<SetMigrationContextHook>();

            // Add migration action completed hooks
            _planBuilder.Hooks.Add<LogMigrationActionsHook>();

            // Add batch migration completed hooks
            _planBuilder.Hooks.Add<LogMigrationBatchesHook<IUser>>();
            _planBuilder.Hooks.Add<LogMigrationBatchesHook<IProject>>();
            _planBuilder.Hooks.Add<LogMigrationBatchesHook<IDataSource>>();
            _planBuilder.Hooks.Add<LogMigrationBatchesHook<IWorkbook>>();
            _planBuilder.Hooks.Add<LogMigrationBatchesHook<ICloudExtractRefreshTask>>();

            // Load the previous manifest if possible
            var prevManifest = await LoadManifest(manifestPath, cancel);

            // Build the plan
            var plan = _planBuilder.Build();

            // Execute the migration
            var result = await _migrator.ExecuteAsync(plan, prevManifest, cancel);

            _timer.Stop();

            // Save the manifest
            await _manifestSerializer.SaveAsync(result.Manifest, manifestPath);

            PrintResult(result);

            _logger.LogInformation($"Migration Started: {startTime}");
            _logger.LogInformation($"Migration Finished: {DateTime.UtcNow}");
            _logger.LogInformation($"Elapsed: {_timer.Elapsed}");

            Console.WriteLine("Press any key to exit");
            Console.ReadKey();
            _appLifetime.StopApplication();
        }

        public Task StopAsync(CancellationToken cancel) => Task.CompletedTask;

        /// <summary>
        /// Prints the result to console. 
        /// You can replace this with a logging based method of your choice.
        /// </summary>
        /// <param name="result">The migration result.</param>
        private void PrintResult(MigrationResult result)
        {
            _logger.LogInformation($"Result: {result.Status}");

            // Logging any errors from the manifest.
            if (result.Manifest.Errors.Any())
            {
                _logger.LogError("## Errors detected! ##");
                foreach (var error in result.Manifest.Errors)
                {
                    _logger.LogError(error, "Processing Error.");
                }
            }

            foreach (var type in ServerToCloudMigrationPipeline.ContentTypes)
            {
                var contentType = type.ContentType;

                _logger.LogInformation($"## {contentType.Name} ##");

                // Manifest entries can be grouped based on content type.
                foreach (var entry in result.Manifest.Entries.ForContentType(contentType))
                {
                    _logger.LogInformation($"{contentType.Name} {entry.Source.Location} Migration Status: {entry.Status}");

                    if (entry.Errors.Any())
                    {
                        _logger.LogError($"## {contentType.Name} Errors detected! ##");
                        foreach (var error in entry.Errors)
                        {
                            _logger.LogError(error, "Processing Error.");
                        }
                    }

                    if (entry.Destination is not null)
                    {
                        _logger.LogInformation($"{contentType.Name} {entry.Source.Location} migrated to {entry.Destination.Location}");
                    }
                }
            }
        }

        private async Task<MigrationManifest?> LoadManifest(string manifestFilepath, CancellationToken cancel)
        {
            var manifest = await _manifestSerializer.LoadAsync(manifestFilepath, cancel);
            if (manifest is not null)
            {
                ConsoleKey key;
                do
                {
                    Console.Write($"Existing Manifest found at {manifestFilepath}. Should it be used? [Y/n] ");
                    key = Console.ReadKey().Key;
                    Console.WriteLine(); // make Console logs prettier
                } while (key is not ConsoleKey.Enter && key is not ConsoleKey.Y && key is not ConsoleKey.N);

                if (key is ConsoleKey.N)
                {
                    return null;
                }

                _logger.LogInformation($"Using previous manifest from {manifestFilepath}");
                return manifest;
            }

            return null;
        }
    }
}

namespace Csharp.ExampleApplication.Config
{
    public sealed class MyMigrationApplicationOptions
    {
        public EndpointOptions Source { get; set; } = new();

        public EndpointOptions Destination { get; set; } = new();
    }
}


namespace Csharp.ExampleApplication.Config
{
    public class EndpointOptions
    {
        public Uri ServerUrl { get; set; } = TableauSiteConnectionConfiguration.Empty.ServerUrl;

        public string SiteContentUrl { get; set; } = string.Empty;

        public string AccessTokenName { get; set; } = string.Empty;

        // Access token configuration should use a secure configuration system.
        public string AccessToken { get; set; } = string.Empty;
    }
}

{
  "source": {
    "serverUrl": "http://server",
    "siteContentUrl": "",
    "accessTokenName": "my-server-token-name",
    "accessToken": "my-secret-server-pat"
  },
  "destination": {
    "serverUrl": "https://pod.online.tableau.com",
    "siteContentUrl": "site-name",
    "accessTokenName": "my-cloud-token-name",
    "accessToken": "my-secret-cloud-pat"
  }  
}

Resources

Python API Reference : Getting started sample and the complete Python API Reference for comprehensive documentation.
C# API Reference: Getting started sample and the complete C# API Reference for detailed documentation.
Code Samples: Code samples to kickstart your development process.

Source Code

The Tableau Migration SDK is open source. The source code is in our GitHub repo.

Contributing

Refer to this handy contribution guide if you would like to contribute to the Migration SDK.