MongoDB

Applies from 2021.1

Alation has certified the MongoDB with the following driver to accommodate additional features:

  • cdata.jdbc.mongodb.MongoDBDriver.cdata.jdbc.mongodb

Alation will provide the above mentioned CData driver license for MongoDB.

Scope of Support

  • Supported as Custom DB with the CData Driver.

  • Metadata Extraction (MDE)

    -Automated MDE

  • Compose

  • Data Profiling

Ports

Port 27017 must be open.

Required Information

  • JDBC driver used to connect to the database: CData JDBC Driver for MongoDB

  • JDBC URI for the MongoDB data source

JDBC URI

When building the URI, include the following components:

  • Host Name

  • Port number

  • RTK Key - Use the RTK parameter only if you have the RTK provided by Alation.

Optional parameters:

Applies from 2023.1

  • Other

    • NaturalColumnOrdinal - Set this parameter to false to display the column numbers in the catalog after extraction.

    • CacheMetadataTables - Set this parameter to true to extract large metadata.

JDBC URI Format

mongodb://<Host_Name>:<Port>?RTK=<RTK_Key>

Example:

mongodb://10.13.64.245:27017?RTK=444752465641535552425641454E545042424D33323632390000000000000000000000000000414C5800005559475655474E4E464242370000

JDBC URI Format (with Optional Parameters)

Applies from 2023.1

mongodb://<Host_Name>:<Port>?RTK=<RTK_Key>;Other="NaturalColumnOrdinal=false";Other='CacheMetadataTables=True'

Example:

mongodb://10.13.49.247:27017?rtk=44475248565A30343034323333305745425452314131000000000000000000000000000000000000423453483336453100004342354833365832335A34420000;Other="NaturalColumnOrdinal=false";Other='CacheMetadataTables=True'

Compose URI Format

Applies from 2023.1

mongodb://<Host_Name>:<Port>?WriteScheme='RawValue';GenerateSchemaFiles=OnCreate;RTK=<RTK_Key>

Example:

mongodb://10.13.77.9:27017/?WriteScheme='RawValue';GenerateSchemaFiles=OnCreate;rtk=44475248565A30343034323333305745425452314131000000000000000000000000000000000000423453483336453100004342354833365832335A34420000

Limitation

Query Log Ingestion is not supported.

Service Account

Create the service account for MongoDB based on basic or Kerberos authentication:

Basic Authentication

See the following example for creating a MongoDB user:

      db.createUser(
      {
      user: "mongouser",
      roles: [ { role: "root", db: "admin" } ]
      }
      )

.. note::

      As shown in the code snippet, if the DBA has the role of a root, access to all databases is automatically provided. If the role is other than the root, make sure that you have the listDatabases privilege. Refer to the corresponding `MongoDB Documentation <https://docs.mongodb.com/manual/reference/built-in-roles/>`_.

Set Up in Alation

Step 1: Add the CData Driver for MongoDB to Alation

Refer to Add the CData Driver into Alation Instance.

Step 2: Add a New Datasource

Add a new Datasource on the Sources page.

Step 3: Set up the Connection

  1. On the Add a Data Source screen of the wizard, specify:

    • Database Type: Custom DB

    • JDBC URI: URI in the required format. See JDBC URI. Example:

      mongodb://10.13.64.245:27017?RTK=444752465641535552425641454E545042424D33323632390000000000000000000000000000414C5800005559475655474E4E464242370000

    • Select Driver: select the JDBC driver for MongoDB from the Select Driver drop-down list:

      • cdata.jdbc.mongodb.MongoDBDriver.cdata.jdbc.mongodb

    • Properties: Provide the properties as:

      {"AuthDatabase":"admin","SupportsCatalogsInTableDefinitions":"True","SupportsSchemasInTableDefinitions":"True"}

  1. Click Save and Continue. The next wizard screen - Set Up a Service Account - will open.

    Note

    Do not select the Kerberos ‘Use Kerberos’ checkbox.

../../_images/MongoDB_01.png

Step 4: Enter Service Account Credentials

  1. Select Yes.

  2. Provide the username and password of the service account created for Alation.

  3. Click Save and Continue. The next wizard screen, Configure Your Data Source, will open.

Step 5: Configure Your Data Source

Click Skip this Step. After this step, you are navigated to the Settings page of your data source.

Metadata Extraction

Configure and perform metadata extraction and verify the results:

  • In Settings > Custom Settings, set the Catalog Object Definition to Schema.Table:

    ../../_images/MongoDB_02.png

  • In Settings > Metadata Extraction, set up and perform MDE. Refer to Metadata Extraction.

Applies from 2023.1

  • To display the column numbers in the catalog after extraction, add the parameter Other="NaturalColumnOrdinal=false" in the URI. See JDBC URI Format (with Optional Parameters) for more details.

  • For Metadata Extraction of large data, add the parameter Other='CacheMetadataTables=True' in the URI. Make sure that the Disable Bulk Extraction checkbox is disabled. See JDBC URI Format (with Optional Parameters) for more details.

Profiling

Configure and perform Sampling and Profiling :

  • Users can run a sample for an individual table on the Samples tab of the Table Catalog page or profile an individual column on the Overview tab of the Column page.

  • Automatic full and selective Profiling is supported.

  • Use the Per-Object Parameters in Settings tab to specify which objects to profile.

  • Custom query-based Sampling is supported. Custom Query-Based Sampling allows you to provide a custom query for profiling each specific table.

  • Deep Column Profiling (Profiling V2 ) is supported.

Query Log Ingestion

Not supported.

Compose

Log into Compose:

  • Authenticate compose with your MongoDB credentials.

  • Use the Schema.Table format for writing queries.

Applies from 2023.1

  • Add the following parameters to the Compose URI to create and insert queries. See Compose URI Format for more details.

    • GenerateSchemaFiles=OnCreate

    • WriteSchema=’RawValue’

Troubleshooting

Logs to collect/review:

  • For logs related to MDE: taskserver.log, taskserver_err.log.

  • For logs related to Compose: connector.log, connector_err.log.

  • For any other errors: alation-error.log, alation-debug.log