- Created by Michelle Cabaya (Unlicensed), last modified on Sept 16, 2019
You are viewing an old version of this page. View the current version.
Compare with Current View Page History
« Previous Version 17 Next »
When importing data into X Dispatch, it is important to know the structure of an X Import configuration file. There are two types of files, which can be distinguished by their file extension. Datetime fields must use the format “yyyyMMdd HHmm". Before version 1.3.1, entries are case sensitive. Make sure to enter these values as they appear. Comments are indicated by the '#' character and continue to the end of the line and can be at the beginning of a line, or at the end. Configuration Items Configuration items are set using a Key=Value format, where key must be one of: ConcurrentDataFiles: Optional (starting in version 1.2.0) This value is set to the number of concurrent data files to be processed and will tell the system how many parallel processes to attempt to create. If this value is set to -1 it will tell the system to create as many as it can. Can be used in both the XINI and XASN file types. If this value is NOT set, it will default to 1 to mimic current functionality. ConcurrentProcesses: Can be set in the X Stream definition to run multiple XASN/XINI files at the same time. DataMask: Required Specifies the extension of the import files (e.g., xml, txt, *). To disregard the file extension, you may use '*', however, the DataMask item should be set whenever possible to prevent the processing of unintended files. Many times, third-party companies sending the import files will upload the file with a ".tmp" extension and rename the file to the correct extension only once the upload has been completed. If the DataMask is set to '*' these temporary files may be processed before they have been completely transferred. DataPath: Required Directory to monitor for incoming files. If the directory is not on the local machine, the fully qualified URI (\\server1\customer1\inbox) must be used. Inherit: Optional Specifies whether files moved to the “ProcPath” directory should inherit the permissions of that directory when moved. A value of “1” indicates files should inherit permissions, while a value of “0” indicates the file should retain its current permissions (default behavior). Delimiter: Required This specifies the type of file to be processed and must be one of: "Excel" Starting version 1.3.0, the Delimiter value can be set to “Excel” to tell the import process to import Excel formatted files. This process supports Excel 2003 and Excel 2007 or newer files (*.XLS, *.XLSX, *.XLSB and *.XLSM). This functionality will require the installation of the 32 bit Microsoft Access Database Engine 2010 Redistributable. In versions prior to 1.4.3, the CSV file was limited to 201 columns. After 1.4.3, the CSV file can an unlimited number of columns. ErrPath: Required The directory in which to store any error log files for the import. NumLinesSkipped: Optional Indicates how many lines to skip from the top on an imported flat file. This allows you to bypass header rows in data files. PreProcessor: Optional Full path to the preprocessor executable. If running the preprocessor from a batch file, you must include the full path to the specified preprocessor within the batch file and not use a referential path. Example: c:\ximport\plugins\eTracPreProcessor.exe - would run properly from a batch file .\eTracPreProcessor.exe - would not run properly from a batch file PreprocessorPassInstance: Optional 0/1 to enable/disable passing of the database instance to the pre-processor. It defaults to 0 to not pass the parameter so that it will not cause a regression for any existing preprocessors. ProcPath: Required The directory in which to store copies of original import file and the log of XML requests used to place an order. Qualifier: Optional Qualifier defaults to the double-quote character and should almost never be modified. The corner case where Qualifier may need to be specified is if the import file format is a hybrid CSV file where escaped data is surrounded with a character other than the double-quote. This case is exceedingly rare. URL: Required Full URL to the XMLListener.asp file. BaseNode: Optional; Required if using XML and Routed Only The XML node used to indicate the base node. OrderNode: Optional; Required if using XML only The XML node used to indicate orders. If listed, relative to BaseNode. Otherwise, it is an absolute path from the root of the document. See DATA ITEMS (XML) below. ParcelNode: Optional; Required if using XML only The XML node used to indicate parcels. See DATA ITEMS (XML) below. ParcelNode is relative to OrderNode if used in XINI or if BaseNode is specified in routed. Otherwise, it is an absolute path from the root of the document. This node supports XPath statements. These statements are a powerful method to extend the capability of returning data. Example: if you wanted the ParcelNode to find all subnodes that start with “D_87”, you could use an XPath statement such as this: S_MAN[D_88_1="W"]/*[contains(local-name(),'D_87')] This statement only looks at S_MAN nodes that have a D_88_1 node that is set to “W” and then returns all sub nodes that start with “D_87”. StripBeginningQualifier: Optional; XML only If the XML file submitted is wrapped in leading characters, this will prevent X Import from processing the document. This is because having leading characters is not valid XML. In order to modify the file into becoming valid XML, you can specify a mask of characters to strip out from the very beginning of the file if they are found. Note that this could be more than one character. Also, note that this is not a dynamic field, so do not wrap the value in quotes or begin it with 0,. StripBeginningQualifier is available beginning in X Dispatch 12.1.3 X Import. StripEndingQualifier: Optional; Required if using XML only If the XML file submitted is wrapped in trailing characters, this will prevent X Import from processing the document. This is because having trailing characters is not valid XML. In order to modify the file into becoming valid XML, you can specify a mask of characters to strip out from the very end of the file if they are found. Note that this could be more than one character. Also, note that this is not a dynamic field, so do not wrap the value in quotes or begin it with 0,. StripBeginningQualifier is available beginning in X Dispatch 12.1.3 X Import. SkipLinesIfBeginsWith: Optional A regular expression is used to test each line to see if it should be processed. If the regular expression matches the current line, it will be skipped and not be processed. This is only processed for fixed width and delimited imports. It will not be considered in an XML import. Available in XINI beginning with 12.1.4. Available in 12.0.0 for XASN. Here is a good reference for regular expressions: http://www.regular-expressions.info/reference.html That regular expression matches any line that does not begin with a double quote ("). Therefore, any line beginning with something other than a double quote would not be processed by X Import. LogRetentionDays: Required, best practice to set between 3 and 7 days with the max set to 7 days for Cloud, except Amazon. The number of days to keep files that are in the ProcPath (.proc, .original) and ErrPath (.err) directories (EdiProcessedPath (.proc, .original) and EdiErrorsPath (.bad) for EDI). If this setting is omitted from the configuration file the number of days to keep the files defaults to 90. If the value in the configuration file cannot be parsed as an Integer the number of days to keep the files defaults to 90. The minimum allowed value is 5. If the value in the configuration file is less than 5, only files older than 5 days will be deleted. MaxReimport: Required with X Dispatch 14.2.4 or newer The number of times to attempt to re-import orders that failed the attempt. Omitting this item or setting it to 0 will have no change on import functionality. Any numeric value higher than 0 will tell XImport to try X number of times to re-import the failed data before giving up. This feature will only re-import failed orders/stops. Any successfully imported orders/stops will not be re-imported. LocalizedDateTimes: Optional in X Dispatch 18.1 or newer 0/1 (disable/enable) this tells the Import process that the date/times are already localized to the stop location in the ASN file, and not to localize them against the server date/time. This value defaults to 0 (disabled) so that it works as it always has. This allows the user to let the import plugin know that the date/time stamp they are passing in (e.g. "07/27/2018 00:00:00") has already been localized. If the user specifies a date/time with a timezone offset, the system will make use of that offset for the timezone (e.g. "07/27/2018T00:00:00-04:00"). Attempts: Optional in X Dispatch 19.2 or newer Specifies how many times each line is attempted on an import before moving on and throwing an error. Defaults to 1 if not set. If not managed correctly, it could potentially cause substantial processing delays. Before implementing ensure that you fully understand how the element works with the MaxFailed and RetryDelay(ms) to prevent issues with integration processing times. MaxFailed: Optional in X Dispatch 19.2 or newer Specifies how many lines in an import can fail to import before the remainder of the file is failed and set for reimport later. If not set, the import process will continue and all of the failed entries will be put into a new file for reimport. For example, if a CSV file has 100 rows, it processes the first 25 successfully, the MaxFailed is set to 15, and the next 15 lines fail, the remaining 60 lines are failed as well. The new file for reimport attempt will have the remaining 75, including the 15 failed entries, and it will be attempted again based on the MaxReimport. If not managed correctly, it could potentially cause substantial processing delays. Before implementing ensure that you fully understand how the element works with the Attempts and RetryDelay(ms) to prevent issues with integration processing times. RetryDelay(ms): Optional in X Dispatch 19.2 or newer Specifies an amount of time in milliseconds to wait in between failed attempts at importing a line from a file. This is the delay for attempting to process each data set. Delimited files would be the delay for processing each row, XML would be a delay for processing each node based on the Attempts value. Defaults to 0 if not set. If not managed correctly, it could potentially cause substantial processing delays. Before implementing ensure that you fully understand how the element works with the Attempts and MaxFailed to prevent issues with integration processing times. There are three types of Data Items: static, dynamic, and macro. Static data items allow you to populate fields with known values. For example, if all of the deliveries sent in on an ASN will be assigned the same order type, origin address, etc. Static data items can be set by prepending the argument with “0,” followed by the static value. EX: UserID=0,ximport-mckesson Macro data items allow you to populate fields with certain pre-defined macros. Currently, the only supported macros are the following: NOW will return the current timestamp at the time of processing, e.g. “04/20/2008 11:11” AKA "Server Time" This would be the time of the server on which X Import is running. Time may be added and subtracted in minute increments by appending the appropriate math after the “NOW” route keyword. TODAY will return the current date at the time of processing with the time set to 12:00 AM, e.g. “04/20/2008 00:00” AKA "X Dispatch Time" This is the time in X Dispatch via dbo.GETCXTDate(). Time may be added and subtracted in day (whole or fractional) increments by appending the appropriate math after the “TODAY” keyword CXTNOW will return the current date at the time of processing within cxtAsp. This would be useful if the time zone of the system running X Import does not line up with the time zone of X Dispatch. For example, if a customer on Cloud is set for Eastern time but the server running their instance is set to Pacific or Central time. Use of modifiers for this function like with NOW added in X Dispatch 11.2.6. Applies to the following XINI fields: Applies to the Following XASN Fields: If CXTNOW is used in a field outside of what is outlined above, the field will contain the results of NOW instead. There was an issue discovered that prevented CXTNOW from being passed to the system for these fields. This issue was resolved in the import plugin version 1.4.5. Macro data items can also be manipulated to add time offsets. For example, if you receive an ASN for same day delivery and the file comes in at inconsistent times (11 PM one day and 1 AM the next), using the above macros may not be sufficient. For these cases, it is possible to add a time offset and then apply a date transformation. If you want to accept a same day ASN before 3 PM, but roll everything after 3 PM to the next day, you could do the following: The syntax for this type of macro manipulation is in the following form: Item=Macro&[OffsetHours:]OffsetMinutes|Transformation Where: WEEKDAYFLOOR: Behaves similarly to DAYFLOOR. If the time, after adding the OffsetHours and OffsetMinutes, is on a Saturday or Sunday, it will be pushed to the following Monday and then set the resulting time to 00:00 WEEKDAYFLOOR was added in X Dispatch 12.0.6. NOW is a macro that pulls the SERVER time, not the time local to the customer. The server time for Cloud customers is always AZT and the imported files will show the time local to the customer. In the example above, if the customer’s local time is EST you would need to add 3 hours to the syntax example: Dynamic data items allow you to set an item’s value based upon the type of input file. Comments
Data Items
Static Data Items
Macro Data Items
Example
Dynamic Data Items
On This Page:
Related Articles:
- Data Items (Fixed-Width)
- Data Items (Delimited)
- X Import as a Service (Deprecated)
- Chaining Preprocessors
- File Format (Decrypt File) - *.XENCRYPTINI
- Config file functions
- Configuration: X Import Translations
- File Format (General)
- Data Items (XML)
- File Format (On Demand) - *.XINI
- File Format (Routed ASN) - *.XASN
Search related import documentation
- No labels