Page Comparison

...

Date:	Author:	Version:	Changes:	Ext.	Int.	Is in Core	Jira Ref.
25 November 2015	Karuna Thulluri	0.1	JSON File export feature added to File Processor (Not yet completed)			N/A

Description

The purpose of this document is to provide a description of the JSON file export feature of the File Processor framework.

...

2 - Overview

The generation of an a JSON-file is done according to a database configuration where it is possible to configure all the tags that must be printed, their structure and how data can be retrieved to fill their content.

Once the configuration is ready the JSON-producer just needs the data that will be passed as a map.

Code Block

xml

title	xml

Sample JSON file

{"voucher_file" : {"currency" : "XYZ","amount" : "200",
"vouchers" : {
"voucher" : {"id" : "11111","serial_number" : "11111","secret_code" : "11111"}
"voucher" : {"id" : "22222","serial_number" : "22222","secret_code" : "22222"}
"voucher" : {"id" : "33333","serial_number" : "33333","secret_code" : "33333"}
"voucher" : {"id" : "44444","serial_number" : "44444","secret_code" : "44444"}
"voucher" : {"id" : "55555","serial_number" : "55555","secret_code" : "55555"}
}
}
}

In this example the map can contain 2 objects: “HEADER” will contain the billing source object used to fill the <Name> and the <BillingSourceId> tags, then in the map we will also find an object containing the list of records with key “RECORDS” for instancesource object used to fill the "currency" and "amount" key/value pairs. The keys and the methods to retrieve the data from the map and from the objects inside the map will be specified in the configuration table called JSON_TAG_DEF.

2.1 - Encoding

The encoding used to generate the file is UTF-8..

...

The persistent objects introduced in this framework and which are as part of the configuration are JSONTagDef (table JSON_TAG_DEF) and DocumentDef (table DOCUMENT_DEF) and . The configuration also uses the ValueFormatter (table VALUE_FORMATTER).

...

TAG_NAME

Name of the tag. For example Recordcurrency, Codeamount, Name, Voucher, Id, brand etc

MAP_KEY

Key to retrieve the correct component from the map. Example: HEADER or FILEDEF or RECORD_DEF or tag name of the parent record from which a child tag value can be obtained.

HEADER holds the file source record.

FILEDEF holds the current instance of JSONDocumenrFiller.

RECORD_DEF holds the CDR_RECORD_DEF (with CODE = DETAIL) instance set up for this JSON Document export

VALUE_TYPE

Can be FIELD (if the value to print is a field of the map object) or METHOD (if the content is retrieved calling a specific method in the map object obtained via the key set in MAP_KEY).

FIELD_NAME

If the value type is FIELD then this specify the field to print in the JSON tag content.

METHOD_NAME

if the value type is METHOD, then this specify the method to invoke on the map object to retrieve the data to fill the content for the current tag.

JSON_DATA_TYPE

The data type with which the JSON field value should be represented while writing value to file. Possible value are OBJECT, ARRAY, STRING, NUMBER , BOOLEANandBOOLEAN.

OBJECT, ARRAY types can be used only for the parent tag types (i.e JSON_TAG_DEF entries that have children).
STRING, NUMBER, BOOLEAN types can be used only for the JSON_TAG_DEF entries that do not have any children.

Note: JSON Key names will always be surrounded by double quotes. This JSON_DATA_TYPE configuration settingsSTRING, NUMBER, BOOLEAN are applicable only for values, not the Keyskeys.

JSON_DATA_TYPE

value

Description of behavior

Example

OBJECT (Default setting for the JSON_TAG_DEF entries that have children)

Can be used only for the parent tag types.

If this type is chosen, the children of this parent tag will be surrounded by curly braces "{ }"

JSON Tag Voucher is set up as OBJECT type

JSON_TAG_DEF.TAG_NAME = voucher, JSON_DATA_TYPE = OBJECT / empty
Resulting line in JSON file:
"voucher" : { "id" : "12345","serial_number" : "12345","secret_code" : "12345" }

ARRAY

Can be used only for the parent tag types.

If this type is chosen, the children of this parent tag will be surrounded by square brackets "[ ]"

JSON Tag Voucher is set up as OBJECT type

JSON_TAG_DEF.TAG_NAME = voucher, JSON_DATA_TYPE = ARRAY
Resulting line in JSON file:
"voucher" : [ "id" : "12345","serial_number" : "12345","secret_code" : "12345" ]

STRING (Default setting for JSON_TAG_DEF entries that do not have any children)

Can be used only for the JSON_TAG_DEF entries that do not have any children.

Resulting value will be surrounded by double quotes.

JSON_TAG_DEF.TAG_NAME = brand

Actual Field Value

Result After Choosing

JSON_DATA_TYPE = STRING

Resulting key value pair in JSON file

XYZ

"XYZ"

"brand" : "XYZ"

NUMBER

Can be used only for the JSON_TAG_DEF entries that do not have any children.

Resulting value will not have double quotes around. Should be used for presenting numeric type values in JSON file.

JSON_TAG_DEF.TAG_NAME = count

Actual Field Value

Result After Choosing

JSON_DATA_TYPE = NUMBER

Resulting key value pair in JSON file

43

"count" : 43

BOOLEAN

Can be used only for the JSON_TAG_DEF entries that do not have any children.

The boolean Boolean equivalent of the actual field value gets written to the json file. Resulting value will not be surrounded by double quotes

JSON_TAG_DEF.TAG_NAME = exported

Actual Field Value	Result After Choosing JSON_DATA_TYPE = BOOLEAN	Resulting key value pair in JSON file
1	true	"exported" : true
0	false	"exported" : false
abcdefg	false	"exported" : false
Y	true	"exported" : true
True	true	"exported" : true

PARENT_ID

Defines the ID of the parent tag and therefore it is used to define the structure of the JSON-file. The parent ID for the root tag will be null.

FLUSH_CONTENT

if ‘Y’ the content generated in the outputStream will be flushed in the JSON-file when the end tag of the current tag is printed.

FORMATTER_ID

ID of the value_formatter table entry. This can be used to format the JSON field value, before applying the JSON_DATA_TYPE (see subsection below). So, first the Value formatter is applied to the field value, and then the JSON_DATA_TYPE setting is applied.

METHOD_INPUT

In some cases when the METHOD_NAME is invoked we also need some extra information that must be passed in the method. For instance in the cdr generation if we have a list of invoice detail lines inside the record tag we need to retrieve only the invoice detail lines related to that billing record and this can be done by defining in this field the tag name related to the object to pass: in this case it will be “Record”.

METHOD_INPUT_CLASS_NAME

This is the class name of the method_input field (“com.CDRator.billing.rating.BillingRecords” in this example).

SEQ_ORDER

If a tag has more than one child, here we can specify the preferred order to print them.

RELOAD_METHOD

If the tag to print is related to a list (i.e. a list of records) then it is possible to load them in multiple batches. In the JSON file generation when the content of the current batch has been printed then we try to retrieve further elements using the reload method if set.

...

An abstract class DocumentFiller has been defined with an abstract method populate that will generate the dataBeans map according to a context map.

The concrete subclasses sub-classes of DocumentFiller class should contain the specific logic for the generation of the document.

Following Concrete subclasses sub-classes of DocumentFiller can be found in the FileProcessor Integration project: see JSONDocumentFiller . See JSONDocumentFiller for more details

3.3.1 - JSONDocumentFiller

...

Create JSON_TAG_DEF entries for the required JSON file format: A record in the JSON_TAG_DEF defines a single element (key/value pair) of the resulting JSON file. It is possible to define parent tag for the JSON_TAG_DEF entry, and thus it is possible to define the full hierarchy of the JSON elements. Please choose proper FORMATTER_ID (ID of the VALUE_FORMATTER table record, used to format the value before writing it to the JSON file) for each JSON_TAG_DEF entry based on the value type.
Create a DOCUMENT_DEF record with FILLER_CLASS = 'com.CDRator.billing.document.json.JSONDocumentFiller', ROOT_TAG_ID = '<Id of the JSON Root Tag entry from JSON_TAG_DEF entries created for this JSON file format>'
The SQL_LOAD_RECORDS column of the above created DOCUMENT_DEF record must contain the SQL query to obtain the records that should be exported to the file. This SQL string can contain the label <BILLING_SOURCE_ID> which will be replaced with appropriate value at run-time.
If this SQL query string contains the label <BILLING_SOURCE_ID>, it will be replaced with the ID of the current source record currently being processed to create the JSON file. Note: The billing source table name is configured in the column BILLING_SOURCE_TABLE_NAME of the table CDR_FILE_DEF. <BILLING_SOURCE_ID> will be replaced with the ID of the current row being processed from the table configured in the column CDR_FILE_DEF.BILLING_SOURCE_TABLE_NAME.
The SQL_RELOAD_RECORDS column of the above created DOCUMENT_DEF record must contain the SQL query to load the next batch of records with ID greater than the last loaded maximum ID, which are to be exported into the JSON file. This SQL query can contain the tags <BILLING_SOURCE_ID> and <MAX_LOADED_ID> labels which will be replaced with appropriate values at run-time
Create a CDR_FILE_DEF record with CODE ='JSON' , BILLING_SOURCE_TABLE_NAME = 'VOUCHER_EXPORT_SOURCE'<Source table name>, BILLING_SOURCE_CLASS= 'com.CDRator.billing.product.vouchers.export.data.VoucherExportSource', EXPORT<Java class for Billing Source>, EXPORT_TYPE = '<The Export_Type for the Customer this JSON file is generated for>', DOCUMENT_DEF_ID = '<ID of the DOCUMENT_DEF record created in above step>', FILEFILE_NAME_ANME-SUFFIX = 'json'. See the sample configuration supplied for more details on the values used for different columns of CDR_FILE_DEF. Please note that the CDR_PARSER_ID, RECORD_SEPARATOR_ID are not used for JSON file export and hence they can be left empty.
The same A post processor method name, post processor hookpoint key, filename format and/or filename generator method can be used for Flat file generation, XML file and JSON file generation, as described in Voucher Export File Processor (GENERATOR) engineconfigured on this CDR_FILE_DEF as needed.
Create a CDR_RECORD_DEF record with CODE ='DETAIL' , RECORD_TABLE_NAME = 'VOUCHER_EXPORT_RECORD'<DB table name for the records to be exported>, RECORD_CLASS = 'com.CDRator.billing.product.vouchers.export.data.VoucherExportRecord'<Java class name for the Records to be exported>, CDR_FILE_DEF_ID = '<ID of the CDR_FILE_DEF created in above step>'. The JSON file processing engine will only need a CDR_RECORD_DEF entry for DETAIL records. So HEADER, TRAILER record definitions are not needed.
Create a CDR_FILE_PROCESSOR entry with CODE='GENERATOR', CDR_FILE_DEF_ID = '<ID of the CDR_FILE_DEF created in earlier step>', FILE_TRANSMITTER_ID= '<Id of the corresponding FILE_TRANSMITTER entry>' and BRAND_ID = '<Corresponding brandId>'

...

For voucher export, value of DOCUMENT_DEF.SQL_LOAD_RECORDS column should be:

Code Block

language	sql

Select

...

 rec.*

...

 from RECORDS recwhere rec.SOURCE_ID = <BILLING_SOURCE_ID> order by

...

 rec.id;

And value of DOCUMENT_DEF.SQL_RELOAD_RECORDS column should be:

Code Block

language	sql

Select

...

 rec.*

...

 from RECORDS recwhere rec.SOURCE_ID = <BILLING_SOURCE_ID> and

...

 rec.id > <MAX_LOADED_ID> order by

...

 rec.id;

If there are 20000 records, then the JSONDocumentFiller uses the SQL queries configured in the DOCUMENT_DEF record as explained below:

...

Versions Compared

Old Version 2

New Version 3

Key

Description

2 - Overview

2.1 - Encoding

3.3.1 - JSONDocumentFiller