Field Reference Guide

This guide explains the field naming conventions and data types used in IOSetu's output formats.

Field Naming Conventions

IOSetu uses a consistent naming pattern to provide multiple representations of each field value.

Base Field (Formatted Value)

The base field name contains the human-readable, formatted value.

Example:

{
  "Effective Date": "2024-01-15",
  "Policy Number": "POL-123456",
  "Premium Amount": "1250.00"
}

Raw Field (- Raw suffix)

Fields with the - Raw suffix contain the original AL3 fixed-width value exactly as it appears in the source file.

Example:

{
  "Effective Date": "2024-01-15",
  "Effective Date - Raw": "20240115"
}

Use Cases: - Audit trails and compliance - Debugging parsing issues - Verifying data integrity - Reconstructing original AL3 files

Formatted Field (- Formatted suffix)

Fields with the - Formatted suffix contain decoded descriptions for coded values, based on ACORD code tables.

Example:

{
  "Coverage Type": "BB",
  "Coverage Type - Formatted": "Baseboard | Basic"
}

Format: Pipe-separated descriptions when multiple codes are present.

Data Types

Numeric Fields

Integer Values:

{
  "Policy Term": "12",
  "Policy Term - Raw": "12"
}

Decimal Values:

{
  "Premium Amount": "1250.00",
  "Premium Amount - Raw": "0000125000"
}

Signed Values:

{
  "Adjustment Amount": "-50.00",
  "Adjustment Amount - Raw": "-0000005000"
}

Date Fields

Format: YYYY-MM-DD (ISO 8601)

{
  "Effective Date": "2024-01-15",
  "Effective Date - Raw": "20240115"
}

Special Values: - Empty/null dates: "" or null - Invalid dates: Original raw value preserved

Time Fields

Format: HH:MM:SS or HH:MM (24-hour)

{
  "Transaction Time": "14:30:00",
  "Transaction Time - Raw": "143000"
}

Alternate Format (when seconds are not provided):

{
  "Transaction Time": "14:30",
  "Transaction Time - Raw": "1430"
}

Text Fields

Alphanumeric:

{
  "Policy Number": "POL-123456",
  "Policy Number - Raw": "POL-123456"
}

Names (with component parsing):

{
  "Insured Name": "John Michael Doe Jr.",
  "Insured Name - Components": {
    "first": "John",
    "middle": "Michael",
    "last": "Doe",
    "suffix": "Jr."
  }
}

Coded Fields

Fields that use ACORD code tables include both the code and its description.

Example:

{
  "State Code": "CA",
  "State Code - Formatted": "California",
  "Coverage Type": "BBGL",
  "Coverage Type - Formatted": "Baseboard | General Liability"
}

Multi-Code Format: Codes are concatenated, descriptions are pipe-separated.

Special Fields

Synthesized Fields

For unknown or undocumented groups, IOSetu synthesizes fields:

HEADR: 30-byte group header

{
  "HEADR": "1MHG000000000000000000000000",
  "HEADR - Raw": "1MHG000000000000000000000000"
}

RAW: Complete raw data for the group

{
  "RAW": "...",
  "RAW - Raw": "..."
}

Validation Metadata

Not included in field data, but available in response headers and validation endpoints:

Headers: - X-IOSetu-Validation-Status: true or false - X-IOSetu-Error-Count: Number of errors - X-IOSetu-Warning-Count: Number of warnings

Output Format Differences

JSON (Hierarchical)

Preserves parent-child relationships:

{
  "1MHG": {
    "Policy Number": "POL-123456",
    "_children": ["2TRG", "3MTG"]
  },
  "2TRG": {
    "Transaction Type": "NB",
    "_parent": "1MHG"
  }
}

NDJSON (Flattened)

One group per line, no hierarchy:

{"1MHG": {"Policy Number": "POL-123456"}}
{"2TRG": {"Transaction Type": "NB"}}

CSV (Flattened)

Denormalized with group code prefix:

group_code,field_name,value,raw_value
1MHG,Policy Number,POL-123456,POL-123456
2TRG,Transaction Type,NB,NB

Parquet (Columnar)

Optimized for analytics with typed columns:

group_code: string
field_name: string
value: string
raw_value: string
formatted_value: string (nullable)

Best Practices

For Application Integration

1. Use formatted values for display and business logic
2. Store raw values for audit trails
3. Use formatted descriptions for user-friendly displays
4. Validate using metadata before processing

For Data Warehousing

1. Use Parquet format for analytics
2. Store all three representations (base, raw, formatted)
3. Index on group codes for efficient queries
4. Partition by policy number or date

For Compliance

1. Preserve raw values for regulatory requirements
2. Log validation metadata for audit trails
3. Track processing timestamps via response headers
4. Store original file checksums for verification

Examples

See the examples directory for complete output examples demonstrating these conventions.