Writing a schema

The schema says what to extract. Its descriptions are the instructions: the clearer each one, the better the result.

Describe every field

For each field, say what it is, where it is printed, and the form you want it in.

{
  "type": "object",
  "properties": {
    "invoice_number": {
      "type": "string",
      "description": "The invoice number, as printed next to 'Invoice No.' at the top right"
    },
    "issue_date": {
      "type": ["string", "null"],
      "description": "The date the invoice was issued, as YYYY-MM-DD"
    },
    "total_due": {
      "type": "number",
      "description": "The amount due, in dollars, without the currency sign; the grand total, after tax"
    },
    "lines": {
      "type": "array",
      "description": "Every line of the items table, in the order printed, across all pages",
      "items": {
        "type": "object",
        "properties": {
          "description": {"type": "string"},
          "quantity": {"type": ["number", "null"]},
          "amount": {"type": "number", "description": "The line's amount, in dollars"}
        }
      }
    }
  }
}
  • What it is: "the grand total, after tax", not just "total", when a document has several totals.
  • Where it is: "next to 'Invoice No.'", "in the segment results table", "on the signature page".
  • The form: "as printed", "as YYYY-MM-DD", "in millions", "without the currency sign", "digits only".

A field with no description is read as its name suggests.

Lists of rows

Put rows in an array of objects. Every row printed becomes an item, in the order printed, however many there are and however many pages the table runs over. Say in the array's description which rows you want: "every line of the items table", "each director listed in the board section".

Missing values

Every field in the schema comes back. A value the document does not give is null, so let fields be missing where they may be: {"type": ["string", "null"]}. A value is never made up to fill a field.

Values worked out from the document

A field can ask for a value that is not printed but follows from what is: a sum, a count, a difference, an item kept or left out by a rule the document gives. Say exactly how it is worked out, and use the pro tier for documents where it takes reading across many pages.

"margin_over_threshold": {
  "type": "integer",
  "description": "The month's counted total minus the monthly threshold in the aid schedule (negative if short)"
}

Limits

  • The top of the schema is an object (or an array).
  • Up to 1 MB of schema; $ref and $defs are fine.
{
  "type": "object",
  "$defs": {
    "Party": {"type": "object", "properties": {"name": {"type": "string"}, "address": {"type": ["string", "null"]}}}
  },
  "properties": {
    "buyer": {"$ref": "#/$defs/Party"},
    "seller": {"$ref": "#/$defs/Party"}
  }
}

Anything to know about the whole document, not one field, such as "amounts are in thousands", goes in the instructions: see Options and jobs.