Batch Tokenization Overview
Batch Overview
The Batch solution is used to process files via the TokenEx Data Protection Platform. Batch sends and receives file jobs via the Secure File Transfer Protocol (SFTP) on port 22 to process batch files. This process can be used to tokenize or detokenize data elements within the file and can parse most file formats, including comma-separated value (CSV), delimited files, tagged files, standard XML, fixed-width files, and full-width file formats.
TokenEx does offer a standard file format that does not require the creation of a custom file mapping and the associated cost.
The maximum unencrypted/uncompressed file size limit is 2GB.
Standard Batch File Format
Using Batch
To use Batch, the client must upload the file to be processed to the appropriate SFTP directory and folder. Typically, a client will be provisioned folders specifying the type of job such as tokenization, detokenization, or deletion, and each job folder will have two sub-folders (input and output). To setup Batch, a client will need to provide the TokenEx Client Services team with a sample file for each unique file format the client wishes to process via Batch. A programmatic mapping and an identifier will be given to that file format, so that TokenEx can process uploaded files in production.
Batch Directory Structure