Run Manual QA Tests

Edit this page   Submit an issue

After you create your connector and validate it using TDVT, run these manual quality assurance tests to confirm that your connector works as expected.

In this section

Before you begin

Be sure that you complete all the following steps before you begin the manual tests for your connector.

  1. Make sure your computer or virtual machine is running Windows, macOS, or Linux.
  2. Install and run Tableau Desktop and Tableau Server.
  3. Install an ODBC or JDBC driver for your database on Tableau Desktop and Tableau Server.
  4. Set up your connectors following these guidelines.

On Tableau Desktop

On Tableau Server

  1. Set the native_api.connect_plugins_path option. For example: 
     tsm configuration set -k native_api.connect_plugins_path -v D:/tableau_connectors

    If you get a configuration error during this step, try adding the --force-keys option to the end of the command. Note: On Linux, make sure the connectors folder can be accessed by tsm. For example:

     tsm configuration set -k native_api.connect_plugins_path -v /var/tmp/plugins –force-keys
  2. Run the following command to apply the pending configuration changes. This will restart the server. 
     tsm pending-changes apply

    For more information about tsm, see tsm Command-Line Reference in Tableau Server on Windows Help or Tableau Server on Linux Help.

Test your connector with Tableau Desktop

Check company name

If your company name doesn’t display, make sure your company name is defined in the manifest.xml file.

<vendor-information>
    <company name="Company Name"/>
</vendor-information>

Connect to the database with valid credentials

  1. Connect and cancel. After you click your connector, close the window. It should return to the previous window without error.

  2. Make valid entries in each field (Server, Username, Password, Port, and so on) and verify that you can connect. Repeat this step with each supported auth mode.

  3. Verify that the default connection name is correct, and that you can change it.

  4. Verify that you can connect using all supported methods of authentication.

  5. Verify that you can connect to a data source with SSL, if applicable. Select the Require SSL checkbox, and then click Sign In.

Test extracts

  1. Create an extract. Right-click the data source, then click Extract Data. Verify that you can create an extract without errors.

  2. Refresh an extract.

For more information, see Refresh Extracts in Tableau Desktop and Web Authoring Help.

Edit your connection

Change all possible items and verify that changes are applied.

  1. Right-click the data source and click Edit Data Source.
    The worksheet opens in Tableau.
  2. In the left pane, under Connections, click the dropdown menu next to the server name and click Edit Connection.
  3. Change something. For example, change the server.
  4. After you click Sign In, you should see the new server name under Connections.

Connect to the correct database with the wrong credentials

Note that the detailed error message in the box is taken from the driver and has no meaning to the Tableau platform. To pass the test, the error message above the box must say “Invalid username or password”.

[Optional] Test driver version

If you defined a minimum driver version in your connector, test connecting to the correct database with the correct credentials, but with an old driver version installed.

For example, in Mariadb, <driver-version min=’3.0’/> is defined in connection-resolver.tdr. You can install a 2.0 driver to test that the connector does not use the old driver and instead, shows the “Download and install the drivers” link.

Test your connector with Tableau Server

Publish to Tableau Server

Perform the publishing tests listed below. For more information about publishing, see the publishing resources listed at the end of this section.

Find publishing resources

For information about publishing a data source, see Publish a Data Source in the Tableau Desktop and Web Authoring Help.

For information about publishing a workbook, see Comprehensive Steps to Publish a Workbook in the Tableau Desktop and Web Authoring Help.

Test extract refreshes

Refresh the extract on Tableau Server with the connector installed on the server.

Create and open workbooks and data sources on Tableau Server

Create a workbook on Tableau Server with the connector installed on the server:

  1. Sign in to Tableau Server.

  2. Under Explore, click Create.

  3. From the dropdown menu, select Workbook.

  4. Select your connector. In this example, the connector name is MariaDB.

  5. Enter the required information to sign in. Repeat as necessary to test all auth modes.

  6. After you connect to the data source, you should be able to create a workbook and save it on the server.

    Note: Web authoring (creating a connection from the web) is not currently available for all connector superclasses. In those cases, your connector won’t appear on the list of connectors on Tableau Server. If you can publish a workbook or data source using your connector to your server, then your connector is loaded correctly, even if you can’t see it on the list of connectors.

OAuth Connector Test Cases

If your connector supports OAuth Authentication, besides the previous test steps, there are some extra steps you need to verify for OAuth on Tableau Desktop/Server.

Test an OAuth Connector on Tableau Desktop

Connect to the database with OAuth tokens

  1. Enter the required information to sign in.
  2. A default browser should pop up and you will need authenticate yourself through OAuth, you should see a consent screen like this:(It’s a google consent screen for illustration, the actual one you see depends on your Identity Provider)
  3. After you allow access, you should see this landing page on your browser:
  4. Then you can close your browser and you should be able to perform the normal manual test steps here

Test an OAuth Connector on Tableau Server

Prerequisite

Follow this instruction to set up OAuth client for your connector on Server first.

Test refresing OAuth token on Tableau Server

If your connector supports oauth, you need to perform this extra step to make sure Tableau can successfully refresh your token.

  1. Go to user’s server settings page and find the pane for Saved Credentials for Data Sources.

  2. Find your connector in the connector list and click Add button next to it, which will invoke the OAuth flow, authenticate yourself and we will save the OAuth token securely in Tableau Server.

  3. Examine the saved OAuth token. It should contain a username that uniquely identifies you. It can also contain a instanceUrl if your oauthConfig file has OAUTH_SUPPORTS_CUSTOM_DOMAIN enabled.

  4. Click the Test button next to your saved credential, it will try to refresh the accessToken and you should see a success message.

Publish OAuth resource to Tableau Server

The publishing experience for OAuth is different than a username-password connection,

The workbook should publish without errors.

Test a Connector Update

This is the minimum set of manual checks to run through when making an update to your connector, but these tests alone do not guarantee that the update is bug-free. For small or cosmetic changes, or for updates to the connector signature only, the following may be enough, but for larger changes you may be required to run through the full set of manual test or re-run TDVT.

Ensure new update can connect to database with valid credentials Note: this test case is a duplicate of other manual test cases. If you’re running through the full manual test suite for the update this can be skipped.

  1. Open Tableau Desktop, and click on your connector in the connector list

  2. Make valid entries in each field (Server, Username, Password, Port, and so on) and verify that you can connect.

  3. Verify that you can connect using all supported methods of authentication.

  4. After connecting, go to a worksheet and drag some data onto the viz and ensure no errors occur and the viz shows expected content

Ensure workbooks made with previous version of connector can be opened with new workbook

  1. Create a viz using the previous version of the connector. Ensure no errors occur and the viz shows expected content. Save as a .twb file.

  2. Close Tableau Desktop, and open a new instance of Tableau Desktop with the new version of your connector. (Note: Tableau must be restarted for changes to the connector to be picked up.)

  3. Open the .twb file you created with the old version of your connector. Ensure that the reconnect dialog is shown, and that the user is prompted to re-enter their credentials only.

  4. Ensure no errors occur when viewing the viz, and that the viz appears the same as the previous version of the connector