'classify-pro' Service

enaio® 11.0

The 'classify-pro' service transfers documents and data between enaio® and the 'Invoice' subsystem of Insiders Cloud to extract metadata.

The task is to transfer the invoice documents and the extracted data in the same way to a transfer directory for the 'Data/document import' automatic action, as is done by enaio® Classify on-premises. This means that the previous import processes to enaio® can be used with the exception of a few adjustments.

The invoice documents are collected by Insiders Cloud connectors from the configured mailbox or the PEPPOL channel and processed in a similar way to enaio® Classify on-premises.

For optimal processing of documents in Insiders Cloud, matching data (master data) can be transferred to Insiders Cloud via *.csv or *.txt files using the 'classify-pro' service.

The 'classify-pro' service can also be used for transferring paper invoices that are scanned using enaio® capture or enaio® capture-pro in conjunction with enaio® capture-transfer-module to Insiders Cloud for processing.

The 'classify-pro' service checks at a time interval, which can be configured, whether Insider Cloud tasks have the 'Finished' status and transfers all data to a transfer directory for import via the 'Data/document import' automatic action.

The following diagram shows the data flows through the components involved:

Installation of the 'classify-pro' Service

The 'classify-pro' service is installed via enaio® service-manager and integrated into enaio® services-admin.

Integration parameters: Port: 7580/memory: 512 MB/instances: 1

The service needs to be restarted after configuring the 'classify-pro' service via the classifypro-prod.yml configuration file.

Setup for Retrieving E-mails from a Mailbox

The access data for the mailbox is required for setting up the Insider Cloud Tenant. It can be accessed via the Internet and Graph API and should be sent to the customer via the invoices.

The Insiders documentation contains more details on this.

Requirement for Setting Up a Tenant for Insiders Cloud

A tenant for Insiders Cloud is set up via OPTIMAL SYSTEMS.

The following data is required:

Mailbox access data and folders
The following requirements need to be met by the tenant:

Processing of e-invoices
Creation of the PDF view document for XRechnung invoices
Creation of the PDF view document for ZUGFeRD files
Extraction of XRechnung attachments
Processing of table data
Extraction of payment terms
Comparison of order data
1-to-1 relationship between task and e-mail
Saving of e-mails, if requested by customer
Permitted file formats for attachments: *.pdf, *.tiff, *.jpg

Mailbox access data and folders

You need to specify the following parameters when requesting Insiders to set up the Insiders Cloud email-connector:

Parameter	Description
Account	The e-mail address to which the e-mails with the electronic invoices as attachments are sent.
Application ID	ID of the mailing platform for access via the Graph API
Client secret	Secret that is created for the application on the mailing platform
Directory ID (tenant/tenant ID)	ID for the tenant/tenant on the mailing platform
Inbox folder	Name of the folder from which the e-mails are to be retrieved, which is generally: Inbox
Send folder	Name of the folder to which the processed e-mail will be moved (when successful): Target folder
Error folder	Name of the folder to which the processed e-mail is moved in the event of an error, for example: Control folder

Permitted E-mail Attachments

The Insiders Cloud email-connector rejects e-mails that contain unauthorized attachments.

Permitted attachments:

PDF files
Image files as *.tiff and *.jpg
ZUGFeRD files with a valid Factur-X attachment
XRechnung invoices with a valid UBL or CII syntax

Configuration of the 'classify-pro' Service

The service is configured via the classifypro-prod.yml configuration file located in the \config directory of enaio® service-manager.

When the service is started, an attempt is made to establish a connection to the Insiders Cloud Tenant.

The connection status is visible at enaio® services-admin. The http://<host>:port/manage/health endpoint also provides information about the status of the connection to the Insiders Cloud Tenant.

After connecting, a configurable waiting period initialDelay is used to check whether Insiders Cloud tasks are available in the configured state (fetch) and can be transferred. After the transfer, the value of state is changed as configured (change-to). The next test takes place after the waiting period.

The http://<host>:port/api/info endpoint provides information about the last processing.

Configuration parameters for the connection:

Parameter	Description
base-url	URL for the connection to the Insiders Cloud Tenant
insiders:connection	Configuration area for connecting to Insiders Cloud and importing the data.
username	Name of the API user that is disclosed when the tenant is requested by Insiders.
password	Password for the API user, which is disclosed when the tenant is requested by Insiders. The password should be encrypted.
retry-limit	Retries for token refresh or API retry attempts

Example:

insiders:
  connection:
    base-url: "https://<insiders CLOUD domain>"
    username: insiders-api@<doamin name for the insiders CLOUD tenant>
    password: ENC(<encrypted password of the insiders CLOUD tenant)
    retry-limit: 3  # Retry limit for token refresh or API retries
                                                            

Configuration for Transferring Scans to Insiders Cloud

The classifypro-prod.yml file can be configured as follows and used for transferring scanned invoices that were received on paper and scanned using enaio® capture or enaio® capture-pro in conjunction with enaio® capture-transfer-module to Insiders Cloud for processing.

Parameter	Description
cron	Configuration for the time in which the availability of data is checked.
transfer:path	Directory in which the master data files to be transferred are filed.
transfer:one-task-per-document	Controls whether all files are transferred in one task if there are several documents in the directory or whether a separate task is created for each file. Default: false.
transfer:task-prefix	If specified, the task ID is created from this prefix and the directory name. If not specified, a task ID is generated by Insiders Cloud.
transfer:attributes	List of attributes that are imported from the .xml file when the task is created. Other attributes in the .xml file are ignored.

Example:

insiders:
  scan-upload:
  cron: "0 0 3 * * *"
  transfer:
    path: '../classify_temp'
    one-task-per-document: false
    task-prefix: "SCAN_"
    attributes: "$Dpi,$ScanDate,$DateOfReceipt"
                                                            

Batches are provided by enaio® capture in a directory. The directory is monitored and batch data is transferred to Insiders Cloud.

The batches look as follows:

Import.xml file

<?xml version="1.0" encoding="utf-8"?>
<STACK StackID="B0000001_11:12:2024_0005" Priority="4" SubSystem="Invoice" Category="Invoice" ExportVerifierDocuments="False" ExportSupervisorDocuments="False" ShowAllDocumentsInVerifier="False" HasBackSideInfos="False" HasDocumentSeparators="False" HasProcessSeparators="False" ExtractionDisabled="False">
  <ATTRIBUTES>
    <KeyValuePair Key="$Dpi" Value="300" />
    <KeyValuePair Key="$DateOfReceipt" Value="11:12:2024" />
    <KeyValuePair Key="$ScanUser" Value="OSADMIN" />
    <KeyValuePair Key="$StackDir" Value="E:\smartfixExchange\Import\B0000001_11122024_0002" />
  </ATTRIBUTES>
  <IMAGE ImageID="B0000001_11122024_0006.PDF" DocID="0003" LocationID="B0000001_11122024_0006.PDF" />
   <IMAGE ImageID="B0000001_11122024_0007.PDF" DocID="0004" LocationID="B0000001_11122024_0007.PDF:0"/>
   <IMAGE ImageID="B0000001_11122024_0007.PDF" DocID="0004" LocationID="B0000001_11122024_0007.PDF:1" Skipped="true"/>
</STACK>
                                                            

The scans and the *.xml file must be stored together in a directory below the configured transfer:path directory.

Example:

After the import, this directory is moved to one of the subdirectories finished or error:

Configuration for Retrieving Insiders Cloud Invoice Data

The configuration is carried out via the classifypro-prod.yml file located in the \config directory of enaio® service-manager.

Configuration parameters
Parameter	Description
cron	Configuration for the time in which the availability of data is checked. The value '-' deactivates the check.
check-prefix	If a value is specified, only tasks with an ID beginning with this prefix are processed.
check-state	List of Insiders Cloud task 'state' values whose data is to be retrieved. Default: finished
check-category	If a value is specified, only tasks from this category are processed. Multiple values are separated by a comma.
check-subsystem	If a value is specified, only tasks from this subsystem are processed. Multiple values are separated by a comma.
success-state	Value for the task state within Insiders Cloud if processing was successful. If there is no value, the previously existing status remains. Default Exporter
submit-error	An error can be reported to Insiders Cloud for a task if an error occurs when processing the task. Otherwise, the task remains in the current status. Default: false
transfer:mode	Specifies how the data is to be transferred to enaio®. Possible values are: file: The data is saved as files in the configured directory.
transfer:export-only-verified	Controls whether only verified values should be written to the transfer file. Default: false
transfer:visual-original	Controls whether the file in 'OSCUST_VISUAL_PATH' can also be the original input file if it is a PDF. A PDF, which is selected as 'Primary', is always generated in the current configuration of Insiders Cloud. This is referenced in 'OSCUST_VISUAL_PATH'. If the original file is also a PDF and this parameter has the value true, then the original file is also referenced and both file names are separated with ';'. The default is 'false'.
transfer:original-email	Controls whether an attempt is always made to file the original e-mail. 'OriginalFiles/Mail_Original.eml' file, type 'other'. To do so, a corresponding setting must be made in the Insiders Cloud. Default: false
transfer:file:path	Transfer directory in which the files are saved. The directory is used as the import directory for the 'Data/document import' automatic action.
transfer:file:prefix	Prefix for the file in which the transfer data such as original invoice files, view documents as a PDF for XRechnung invoices and the extracted invoice data are saved.
transfer:file:trigger-file-suffix	If a value is specified here, a file with the specified suffix is created for each exported *.xml file. If file-in-folder is also activated, the suffix is added to the directory name. Default: .ISU_IMPORT_START
transfer:file:exclude-altered-by	Controls whether the parameter 'altered_by' should not be included in the metadata file. Default: true
transfer:file:one-file-per-document	Controls whether several documents are placed in an export file if there are several documents in a task or whether each export file contains a maximum of one document. The name of the export files then consists of the task ID and a sequential number. If file-in-folder is activated, this setting has no effect, as an export.xml file is always created in the export directory. Default: false
transfer:file:file-in-folder	Controls the name and location of the *.xml export file. It is either situated next to the directory with the documents with the same name and the extension xml. Or it is filed in the directory with the documents under the name export.xml. Default: false

Example:

insiders:
  task-download:
    #cron: '-' # no action
    cron: '0 */1 * * * *'
    check-prefix: ''
    check-state: "Finished"
    #check-category: "Invoice"
    #check-subsystem: "Invoice"
    # leave empty to keep state:
    success-state: 'Exporter'
    submit-error: false
    transfer:
      #mode: file/plugin
      mode: file
      export-only-verified: false
      # if original is also a PDF also reference as visual (and not only as "original"). Primary is always referenced as visual
      visual-original: false
      original-email: true
      file:
        path: '<folder that is shared the enaio importer>'
        prefix: 'CLASSIFY_'
        trigger-file-suffix: '.ISU_IMPORT_START'
        exclude-altered-by: true
        one-file-per-document: false
        file-in-folder: true
                                                            

Storage of Data in the Transfer Directory

An *.xml file with the task ID as the file name is saved in the transfer directory for each Insiders Cloud task.

Example:

<STACK Category="Invoice" SubSystem="Invoice" StackID="MAIL_20250123_000034">
    <ATTRIBUTES>
        <KeyValuePair Key="$AmountCluster" Value="Cluster 02: < 2.5K"/>
        <KeyValuePair Key="$InvoiceType" Value="FI"/>
        <KeyValuePair Key="$IsEInvoice" Value="1"/>
        <KeyValuePair Key="$VendorName" Value="Testverkäufer"/>
        <KeyValuePair Key="$VendorNo" Value=""/>
    </ATTRIBUTES>
    <PROCESS ProcessID="0000">
        <DOCUMENT DocID="0000" DocClass="Invoice/eInvoice" AlteredBy="System" ExportName="eInvoice" comment="">
            <PAGE LocationColorID="03.06a-INVOICE_ubl.xml.pdf:0" Attached="FALSE" BackSide="FALSE" Empty="FALSE" ImageID="03.06a-INVOICE_ubl.xml.pdf:0" LocationID="03.06a-INVOICE_ubl.xml.pdf:0" Path="D:\enaio\service-manager\temp\insiders\CLASSIFY_MAIL_20250123_000034\03.06a-INVOICE_ubl.xml.pdf:0"/>
            <PAGE LocationColorID="03.06a-INVOICE_ubl.xml.pdf:1" Attached="FALSE" BackSide="FALSE" Empty="FALSE" ImageID="03.06a-INVOICE_ubl.xml.pdf:1" LocationID="03.06a-INVOICE_ubl.xml.pdf:1" Path="D:\enaio\service-manager\temp\insiders\CLASSIFY_MAIL_20250123_000034\03.06a-INVOICE_ubl.xml.pdf:1"/>
            <PAGE LocationColorID="03.06a-INVOICE_ubl.xml.pdf:2" Attached="FALSE" BackSide="FALSE" Empty="FALSE" ImageID="03.06a-INVOICE_ubl.xml.pdf:2" LocationID="03.06a-INVOICE_ubl.xml.pdf:2" Path="D:\enaio\service-manager\temp\insiders\CLASSIFY_MAIL_20250123_000034\03.06a-INVOICE_ubl.xml.pdf:2"/>
            <PAGE LocationColorID="03.06a-INVOICE_ubl.xml.pdf:3" Attached="FALSE" BackSide="FALSE" Empty="FALSE" ImageID="03.06a-INVOICE_ubl.xml.pdf:3" LocationID="03.06a-INVOICE_ubl.xml.pdf:3" Path="D:\enaio\service-manager\temp\insiders\CLASSIFY_MAIL_20250123_000034\03.06a-INVOICE_ubl.xml.pdf:3"/>
            <PAGE LocationColorID="03.06a-INVOICE_ubl.xml.pdf:4" Attached="FALSE" BackSide="FALSE" Empty="FALSE" ImageID="03.06a-INVOICE_ubl.xml.pdf:4" LocationID="03.06a-INVOICE_ubl.xml.pdf:4" Path="D:\enaio\service-manager\temp\insiders\CLASSIFY_MAIL_20250123_000034\03.06a-INVOICE_ubl.xml.pdf:4"/>
            <FIELDGROUP Name="default">
                <FIELD Rating="OK" Name="INV_AMOUNT" Value="1804.0" VerifierAction="NoAction" Checks="OK"/>
                <FIELD Rating="OK" Name="INV_COMPLIANCE_COMMENTS" Value="Indicator for tax exemption is not on the document (compliance check)" VerifierAction="NoAction" Checks="OK"/>
                <FIELD Rating="OK" Name="INV_COMPLIANT" Value="false" VerifierAction="Changed" AlteredBy="insiders-admin@optimal-systems-" Checks="OK"/>
                <FIELD Rating="OK" Name="INV_CREDIT_NOTE" Value="false" VerifierAction="NoAction" Checks="OK"/>
                <FIELD Rating="OK" Name="INV_CURRENCY" Value="EUR" VerifierAction="NoAction" Checks="OK"/>
                <FIELD Rating="OK" Name="INV_DATE" Value="2021-04-23" VerifierAction="NoAction" Checks="OK"/>
                <FIELD Rating="OK" Name="INV_DELIVERY_DATE" Value="2021-04-23" VerifierAction="OCR" AlteredBy="insiders-admin@optimal-systems-" Checks="OK"/>
                <FIELD Rating="OK" Name="INV_DELIVERY_MONTH" Value="04.2021" VerifierAction="AutoFill" Checks="OK"/>
                <FIELD Rating="OK" Name="INV_DELIVERY_WEEK" Value="16.2021" VerifierAction="AutoFill" Checks="OK"/>
                <FIELD Rating="OK" Name="INV_LEITWEG_ID" Value="991-33333TEST-33" VerifierAction="NoAction" Checks="OK"/>
                <FIELD Rating="OK" Name="INV_NET_AMOUNT1" Value="1600.0" VerifierAction="NoAction" Checks="OK"/>
                <FIELD Rating="OK" Name="INV_NET_AMOUNT2" Value="-100.0" VerifierAction="NoAction" Checks="OK"/>
                <FIELD Rating="OK" Name="INV_NUMBER" Value="112233" VerifierAction="NoAction" Checks="OK"/>
                <FIELD Rating="OK" Name="INV_TAX_AMOUNT1" Value="304.0" VerifierAction="NoAction" Checks="OK"/>
                <FIELD Rating="OK" Name="INV_TAX_AMOUNT2" Value="0.0" VerifierAction="NoAction" Checks="OK"/>
                <FIELD Rating="OK" Name="INV_TAX_RATE1" Value="19.0" VerifierAction="NoAction" Checks="OK"/>
                <FIELD Rating="OK" Name="INV_TAX_RATE2" Value="0.0" VerifierAction="NoAction" Checks="OK"/>
                <FIELD Rating="OK" Name="RE_CITY" Value="Testhausen" VerifierAction="Confirmed" AlteredBy="insiders-admin@optimal-systems-" Checks="OK"/>
                <FIELD Rating="OK" Name="RE_COUNTRY" Value="DE" VerifierAction="Confirmed" AlteredBy="insiders-admin@optimal-systems-" Checks="OK"/>
                <FIELD Rating="OK" Name="RE_NAME" Value="Testkäufer" VerifierAction="Confirmed" AlteredBy="insiders-admin@optimal-systems-" Checks="OK"/>
                <FIELD Rating="OK" Name="RE_RECIPIENT_NO" VerifierAction="Confirmed" AlteredBy="insiders-admin@optimal-systems-" Checks="OK"/>
                <FIELD Rating="OK" Name="RE_STREET" Value="Teststraße " VerifierAction="Confirmed" AlteredBy="insiders-admin@optimal-systems-" Checks="OK"/>
                <FIELD Rating="OK" Name="RE_TAX_ID_NO" VerifierAction="Confirmed" AlteredBy="insiders-admin@optimal-systems-" Checks="OK"/>
                <FIELD Rating="OK" Name="RE_VAT_ID_NO" VerifierAction="Confirmed" AlteredBy="insiders-admin@optimal-systems-" Checks="OK"/>
                <FIELD Rating="OK" Name="RE_ZIPCODE" Value="12345" VerifierAction="Confirmed" AlteredBy="insiders-admin@optimal-systems-" Checks="OK"/>
                <FIELD Rating="OK" Name="VE_CITY" Value="Testhausen" VerifierAction="Confirmed" AlteredBy="insiders-admin@optimal-systems-" Checks="OK"/>
                <FIELD Rating="OK" Name="VE_COUNTRY" Value="DE" VerifierAction="Confirmed" AlteredBy="insiders-admin@optimal-systems-" Checks="OK"/>
                <FIELD Rating="OK" Name="VE_IBAN" VerifierAction="Confirmed" AlteredBy="insiders-admin@optimal-systems-" Checks="OK"/>
                <FIELD Rating="OK" Name="VE_NAME" Value="Testverkäufer" VerifierAction="Confirmed" AlteredBy="insiders-admin@optimal-systems-" Checks="OK"/>
                <FIELD Rating="OK" Name="VE_STREET" Value="Teststraße " VerifierAction="Confirmed" AlteredBy="insiders-admin@optimal-systems-" Checks="OK"/>
                <FIELD Rating="OK" Name="VE_SWIFT_BIC" VerifierAction="Confirmed" AlteredBy="insiders-admin@optimal-systems-" Checks="OK"/>
                <FIELD Rating="OK" Name="VE_TAX_ID_NO" Value="1121081508150" VerifierAction="OCR" AlteredBy="insiders-admin@optimal-systems-" Checks="OK"/>
                <FIELD Rating="OK" Name="VE_VAT_ID_NO" VerifierAction="OCR" AlteredBy="insiders-admin@optimal-systems-" Checks="OK"/>
                <FIELD Rating="OK" Name="VE_VENDOR_NO" VerifierAction="Confirmed" AlteredBy="insiders-admin@optimal-systems-" Checks="OK"/>
                <FIELD Rating="OK" Name="VE_ZIPCODE" Value="12345" VerifierAction="Confirmed" AlteredBy="insiders-admin@optimal-systems-" Checks="OK"/>
                <FIELD Rating="OK" Name="#OSCUST_IMPORT_ID" Value="38c1457f-f11c-4eee-836f-ff6536c37f0c" VerifierAction="NoAction" Checks="OK"/>
                <FIELD Rating="OK" Name="#OSCUST_ORIGINAL_PATH" Value="D:\enaio\service-manager\temp\insiders\CLASSIFY_MAIL_20250123_000034\Original\03.06a-INVOICE_ubl.xml" VerifierAction="NoAction" Checks="OK"/>
                <FIELD Rating="OK" Name="#OSCUST_VISUAL_PATH" Value="D:\enaio\service-manager\temp\insiders\CLASSIFY_MAIL_20250123_000034\Primary\03.06a-INVOICE_ubl.xml.pdf" VerifierAction="NoAction" Checks="OK"/>
                <FIELD Rating="OK" Name="#OSCUST_EMAIL_PATH" Value="D:\enaio\service-manager\temp\insiders\CLASSIFY_MAIL_20250123_000034\Primary\03.06a-INVOICE_ubl.eml" VerifierAction="NoAction" Checks="OK"/>
              </FIELDGROUP>
        </DOCUMENT>
    </PROCESS>
</STACK>
                                                            

The transferred attachments of an e-mail are referenced in a FIELD tag with #OSCUST_ORIGINAL_PATH and #OSCUST_VISUAL_PATH. If several files of type XML and PDF have been transferred, they are listed in the respective tag name separated by a semicolon.

A unique ID for each 'PROCESS' is entered in the #OSCUST_IMPORT_ID FIELD tag. If the one-file-per-document parameter has been set, a PROCESS with several documents is split into several *.xml files. When processing these files, they can be assigned to a PROCESS again via the #OSCUST_IMPORT_ID.

As a minimum, the subdirectory 'Original' is created, in which the non-'electronic invoices' are saved as a PDF, as well as the electronic invoice of type ZUGFeRD as a PDF with an embedded *.xml file and the XRechnung invoices as an *.xml file.

In the case of an XRechnung invoice, the 'Primary' subdirectory, which contains the PDF document for viewing, is created.

If the tenant has been configured so that the e-mail can also be retrieved, it is saved in the 'Other' directory.

Notes on Insiders Cloud Subsystems

The 'classify-pro' service is optimized for the Insiders Cloud subsystem 'Invoice'. It is possible that data from other subsystems of a tenant can also be retrieved and processed in the same way as the Insiders Cloud subsystem 'Invoice'.

Adaptations for special processing will need to be implemented in the project.

Configuration for the Transfer of Master Data to the Insiders Cloud Tenant

Insiders Cloud offers a range of master data tables that can be populated via REST API. The master data is provided from the customer's ERP systems via export files (*.csv or *.txt) in a transfer directory.

The 'classify-pro' service checks this directory at configured intervals and reads the data of existing files and transfers them to the relevant tenant table in chunks via a REST call.

After processing, the file is moved to a fixed directory below the configured master data import directory:

error: Error
warning: Data had to be shortened
finished: Successful

A log file is also created in the error and warning directory.

The start and end of processing is logged in the service log.

The following parameters for the master data tables SFI_RECIPIENT and SFI_VENDOR can be configured in the classify-prod.yml file for the transfer of master data:

Configuration parameters:
- name	Name of the Insiders Cloud master data table. Any number of tables can be listed. You will find a sample configuration for the SFI_RECIPIENT and SFI_VENDOR tables below.
chunk-size	The number of data records to be transferred in a transaction. The log file summarizes the results of all transactions for the master data file. If a chunk is rejected by Insiders Cloud due to errors, it is processed in single-line processing. If an individual chunk runs into an error, this is logged in the log file with its number in the overall file.
csv	Details of the *.csv file from which the data is imported
csv:encoding	'UTF-8', UTF-16BE, UTF-16LE, UTF-16, and 'ISO-8859-1' are available as values. 'UTF-8' is used as the default if an incorrect value is entered.
csv:separator	Separator used for the column values in the master data file, such as 'TAB' for tabulator or characters such as comma, semicolon, or asterisk ('*').
csv:start-line	The number of the line for the first data record to be read.
file-prefix	It is advisable to give the file names a prefix so that there is a reference to the master data table.
insiders:masterdata-upload	Configuration area for the transfer of master data from a .csv or .txt file to the tables of the Insiders Cloud Tenant.
mapping	Configuration area for mapping the columns in the *.csv file to the columns of the Insiders Cloud table.
mapping:col	The number of the column in the *.csv file.
mapping:default	The value to be set in the Insiders Cloud table if no value is contained in the *.csv file or if the column is not contained.
mapping:format	Specification to adjust the values in the import file for the import. Available values are: TRIM: Remove spaces at the beginning and end. TRIMALL: Remove all spaces SIMPLE: Remove all characters except letters and numbers. Example: 25$ and twenty cents! → 25andtwentycents INTEGER(<decimal separator>): Import numerical value before the decimal separator. Example: 1,123.00 → 1123 DECIMAL(<decimal separator>): Import decimal number value without superfluous zeros and characters. The separator for import is a decimal point. Example: 1,222.2500 → 1222.25 DATE(<source format>): Convert the date value from the specified source format to the format "dd.MM.yyyy". Example: DATE(yyyy-MM-dd): 2024-12-31 → 31.12.2024 The specifications for decimal separators and the date format for INTEGER, DECIMAL, and DATE refer to the format in the source file. This is converted into the format required for the import. Only one value may be used.
mapping:length	The number of characters to which the value should be truncated. If there is no value, then there is no truncation.
mapping:target	Name of the column in the Insiders Cloud table
reset-data	Controls whether the existing data should be replaced or supplemented. Default: true
subsystem	There are different master data tables in the different subsystems in Insiders Cloud. Currently, only the 'Invoice' subsystem is supported.
transfer:cron	Configuration for the time in which the availability of data is checked. The value '-' deactivates the check.
transfer:path	Directory in which the master data files to be transferred are filed.
transfer:tables	Area for the configuration of all tables whose data is to be transferred.

Example:

insiders:
  masterdata-upload:
    #cron: '-' # no action
    cron: '0 */1 * * * *'
    transfer:
      path: "input"
      tables:
        - name: SFI_RECIPIENT
          file-prefix: SFI_RECIPIENT
          reset-data: true
          chunk-size: 100
          subsystem: Invoice
          csv:
            separator: "*"
            start-line: 2
            encoding: ISO-8859-1
          mapping:
            - col: 2
              target: RE_PK
              length: 20
            - col: 3
              target: RE_RECIPIENT_NO
              length: 20
              #format possible values: INTEGER(<separator>), DECIMAL(<separator>), TRIM, TRIMALL, SIMPLE, DATE
              format: SIMPLE
            - col: 1
              target: RE_NAME
              length: 80
            - col: 7
              target: RE_STREET
              length: 35
            - col: 4
              target: RE_COUNTRY
              length: 2
              default: DE
            - col: 6
              target: RE_ZIPCODE
              length: 10
            - col: 5
              target: RE_CITY
              length: 35
        - name: SFI_VENDOR
          file-prefix: SFI_VENDOR
          reset-data: true
          csv:
            separator: "TAB"
            start-line: 6
            encoding: UTF-8
          mapping:
            - col: 1
              target: VE_RECIPIENT_NO
              length: 20
            - col: 2
              target: VE_VENDOR_NO
              length: 20
            - col: 3
              target: VE_NAME
              length: 80
            - col: 4
              target: VE_STREET
              length: 35
            - col: 5
              target: VE_COUNTRY
              length: 2
            - col: 6
              target: VE_VAT_ID_NO
              length: 20
            - col: 7
              target: VE_BANK_NO
              length: 15
            - col: 8
              target: VE_BANK
              length: 60
            - col: 9
              target: VE_ZIPCODE
              length: 10
            - col: 10
              target: VE_CITY
              length: 35
            - col: 11
              target: VE_SWIFT_BIC
              length: 11
            - col: 12
              target: VE_ACCOUNT_NO
              length: 18
            - col: 13
              target: VE_IBAN
              length: 34
                                                            

Configuration for Logging Actions for Evaluation by the Monitoring Application

Monitoring applications that evaluate service log files are generally used to visualize processing operations.

The 'classify-pro' service writes its actions to a log file, which is configured as follows in the classifypro-prod.yml file.

Example:

monitor:
  file: '../../logs/classifypro.log'
  max-file-size: 100MB
  total-size-cap: 3GB
  max-days: 3
  details: true
                                                            

Configuration parameters:
total-size-cap	Maximum size of all log files.
max-file-size	Maximum size of a log file.
max-days	Maximum retention length of log files.
file	Target file for the monitor log. The log file is renamed and restarted daily.
details	This specification controls whether further system status data is queried and logged. Example: Determine the total number of tasks before downloading tasks and query the number in the tables after uploading master data. Default: false

The actions of the transfer are logged

by the Insiders Cloud tasks to the transfer directory of the 'Data/document import' automatic action,
from the transfer directory for master data to Insiders Cloud,
from the transfer directory for enaio® capture and enaio® capture-pro to Insiders Cloud.

Example of a log file:

12:00.008 [scheduling-1] INFO  MONITOR [TASK-DOWNLOAD] - *** Start download tasks in state: Finished
12:00.143 [scheduling-1] INFO  MONITOR [TASK-DOWNLOAD] - available tasks: 58
12:00.143 [scheduling-1] INFO  MONITOR [TASK-DOWNLOAD] -  - Coordinator: 1
12:00.143 [scheduling-1] INFO  MONITOR [TASK-DOWNLOAD] -  - Supervisor: 1
12:00.143 [scheduling-1] INFO  MONITOR [TASK-DOWNLOAD] -  - Verifier: 56
12:00.143 [scheduling-1] INFO  MONITOR [TASK-DOWNLOAD] - tasks to process: 0
...
10:00.385 [scheduling-1] INFO  MONITOR [MASTERDATA] - check masterdata files in C:\DEV\git\classifyservice\..\classify_temp\masterdata
10:00.385 [scheduling-1] INFO  MONITOR [MASTERDATA] - Processing file VENDOR.txt ...
10:01.284 [scheduling-1] INFO  MONITOR [MASTERDATA] - uploaded 2 'SFI_VENDOR', errors: 0
10:01.286 [scheduling-1] INFO  MONITOR [MASTERDATA] - Move file to ..\classify_temp\masterdata\finished\VENDOR.txt
...
10:00.380 [scheduling-1] INFO  MONITOR [TASK-UPLOAD] - check scan files in C:\DEV\git\classifyservice\..\classify_temp\scanupload
10:00.380 [scheduling-1] INFO  MONITOR [TASK-UPLOAD] - processing file 'C:\DEV\git\classifyservice\..\classify_temp\scanupload\14012025_13\import.xml'
10:00.380 [scheduling-1] INFO  MONITOR [TASK-UPLOAD] - error processing C:\DEV\git\classifyservice\..\classify_temp\scanupload\14012025_13
10:00.380 [scheduling-1] INFO  MONITOR [TASK-UPLOAD] - processing file 'C:\DEV\git\classifyservice\..\classify_temp\scanupload\14012025_09\import.xml'
10:00.380 [scheduling-1] INFO  MONITOR [TASK-UPLOAD] - created task: ea9510df
10:00.380 [scheduling-1] INFO  MONITOR [TASK-UPLOAD] - upload done: uploaded 1 tasks (2 documents), errors: 1
                                                            

Data Protection in Insiders Cloud

Initially, Insiders delivers at least one user with a password to OPTIMAL SYSTEMS and OPTIMAL SYSTEMS to each sub-tenant. As long as the password is known to OPTIMAL SYSTEMS or Insiders, everything can be configured with the account according to these user rights, which usually requires administrator rights. This is no longer possible once the new user changes the password.

It is then only possible to view the throughput statistics with the administrator of the parent tenant and the batches in the coordinator. Batches can be updated and reset there and logs can be imported.

Access to the decrypted images and the matching data is not possible.

The same applies if the sub-tenant switches to its own AD (SSO).