Pipeline Menu (Beta)

icon
Pipeline Help
Folder Scanning Help
Open a Ticket here if you have any issues or suggestions

Pipeline Configuration and Usage Tutorial

  • Configure the pipeline config file and input files to run files against it
  • For reuse, download the config file and re-upload it when needed, or place it in /pipeline/defaultWebUIConfigs/ to auto-load in the web UI for all users

Steps to Configure and Use Your Pipeline

  1. Access Configuration

    • Upon entering the screen, click on the Configure button.

  2. Enter Pipeline Name

    • Provide a name for your pipeline in the designated field.

  3. Select Operations

    • Choose the operations for your pipeline (e.g., Split Pages), then click Add Operation.

  4. Configure Operation Settings

    • Input the necessary settings for each added operation. Settings are highlighted in yellow if customization is needed.

  5. Add More Operations

    • You can add and adjust the order of multiple operations. Ensure each operation's settings are customized.

  6. Save Settings

    • Click Save Operation Settings after customizing settings for each operation.

  7. Validate Pipeline

    • Use the Validation button to check your pipeline. A green indicator signifies correct setup; a pop-out error indicates issues.

  8. Download Pipeline Configuration

    • To use the configuration for folder scanning (or save it for future use and reupload it), you can also download a JSON file in this menu. You can also pre-load this for future use by placing it in /pipeline/defaultWebUIConfigs/. It will then appear in the dropdown menu for all users to use.

  9. Submit Files for Processing

    • If your pipeline is correctly set up close the configure menu, input the files and hit Submit.

  10. Note on Web UI Limitations

    • The current web UI version does not support operations that require multiple different types of inputs, such as adding a separate image to a PDF.
Current Limitations
  • Cannot have more than one of the same operation
  • Cannot input additional files via UI
  • All files and operations run in serial mode

User Guide for Local Directory Scanning and File Processing

  1. Setting Up Watched Folders:

    • Create a folder where you want your files to be monitored. This is your 'watched folder'.
    • The default directory for this is ./pipeline/watchedFolders/
    • Place any directories you want to be scanned into this folder, this folder should contain multiple folders each for their own tasks and pipelines.

  2. Configuring Processing with JSON Files:

    • In each directory you want processed (e.g ./pipeline/watchedFolders/officePrinter), include a JSON configuration file.
    • This JSON file should specify how you want the files in the directory to be handled (e.g., what operations to perform on them) which can be made, configured and downloaded from Indigo-PDF Pipeline interface.r

  3. Automatic Scanning and Processing:

    • The system automatically checks the watched folder every minute for new directories and files to process.
    • When a directory with a valid JSON configuration file is found, it begins processing the files inside as per the configuration.

  4. Processing Steps:

    • Files in each directory are processed according to the instructions in the JSON file.
    • This might involve file conversions, data filtering, renaming files, etc. If the output of a step is a zip, this zip will be automatically unzipped as it passes to next process.

  5. Results and Output:

    • After processing, the results are saved in a specified output location. This could be a different folder or location as defined in the JSON file or the default location ./pipeline/finishedFolders/.
    • Each processed file is named and organized according to the rules set in the JSON configuration.

  6. Completion and Cleanup:

    • Once processing is complete, the original files in the watched folder's directory are removed.
    • You can find the processed files in the designated output location.

  7. Error Handling:

    • If there's an error during processing, the system will not delete the original files, allowing you to check and retry if necessary.

  8. User Interaction:

    • As a user, your main tasks are to set up the watched folders, place directories with files for processing, and create the corresponding JSON configuration files.
    • The system handles the rest, including scanning, processing, and outputting results.