Create Ingestion

The Ingestion service is built to automate data flows between systems. It helps manage, orchestrate, and automate the movement of data between different systems easily and efficiently, while providing the ability to track, monitor, and manage data flows.

To create an Ingestion service, follow these steps:

Step 1: In the menu bar, select Data Platform > select Workspace Management > select Workspace name

Note: Users can access the Ingestion service directly by: selecting Data Platform > Ingestion service from the menu bar

Step 2: In the My Services section, click Create > the New Service popup is displayed, select Ingestion Service > Create

Step 3: In the Ingestion Service creation form, enter the Basic Information:

• Name (required): Service name

Note: The service name must be between 1 and 30 characters. It may contain lowercase letters a-z, uppercase letters A-Z, or digits 0-9.

• Description (optional): Service description
• Version (required): select a version

Step 4: Click Next Step to move to the Node Configuration screen

• Type: Select the configuration type for the service
• Number of node: select the appropriate number of nodes

warning

The number of nodes must be greater than or equal to 1 and less than or equal to 10

• Storage policy: select a storage policy
• Disk (GB): enter the disk size

warning

The disk size must be greater than or equal to 100 and less than or equal to 1000

Step 5: Click Next Step to move to the Advance screen

• Enter Mount storage information
• Name: Storage name
• Path: path to the folder in the storage

Users can add more Mount storage entries by clicking the "+" sign

warning

You can add up to 5 Mount Storage entries

• Enter Nars storage information
  • Bucket name (required): bucket name
  • Endpoint (required): access address
  • Access key (required): access key
  • Secret (required): access password
  • Path (required): storage folder path

• Single Sign On:

  • If Single Sign On is not checked, Superset is initialized with Basic authen authentication
  • If Single Sign On is checked:
    • Provider: FPT ID - Users enter the following information:
      • Username: username
      • Email: FPT email address

  

  • Provider: Google - Users enter the following information:
    • Client ID: an ID code used to authenticate the client with Google
    • Client Secret: password used to authenticate the client with Google
    • Email: email address

  

  • Provider: Keycloak - Users enter the following information:
    • Auth Provider name: provider name
    • Realm: a management space in which all users, groups, roles, clients, and other objects are managed and secured independently
    • Auth server url: the base URL of the Keycloak server, used by clients to perform authentication
    • Client ID: an ID code used to authenticate the client with Keycloak
    • Client Secret: password used to authenticate the client with Keycloak
    • Username: username in Keycloak
    • Email: email address in Keycloak

  

Custom Domain

• Purpose: Allows configuring a custom domain to access services.
• For Public Workspace: Used to assign a domain and certificate without needing to enable/disable TLS (HTTPS is always available).
• For Private Workspace: In addition to the domain and certificate, users can optionally enable or disable TLS/SSL to decide whether to use HTTPS or HTTP.
  • Workspace is Public
  • Custom domain: Check to enable a custom domain.
  • Domain: Enter the domain name (e.g., abc.local, jupyter.example.com).
  • Certificate name: Select from the list of certificates already imported in Certificate Manager.
  • Buttons:
    • Manage certificate: Opens the certificate management screen.
    • Validate: Checks that the certificate is valid for the domain.

note

For a Public Workspace, the TLS/SSL certificate option is not displayed — the system supports HTTPS by default.

• Workspace is Private
  • Custom domain: Check to enable a custom domain.
  • Domain: Enter the domain name.
  • TLS/SSL certificate: Check to enable HTTPS for services.
  • Certificate name: Select from the list of certificates.
  • Buttons:
    • Manage certificate: Opens certificate management.
    • Validate: Checks the certificate.

note

If TLS/SSL certificate is left unchecked, the service will run over HTTP and no certificate is required.

Step 6: Click Next Step to move to the Review & Create screen

Step 7. Review the entered information, then click Create to complete the initialization of the Ingestion service.

Ingestion service initialization is complete when the Worker Status is Succeeded and the Status of the Ingestion service is Healthy (~10 minutes)