Deep Transform

Deep transform extracts the entities you care about from a document and returns JSON that conforms to a schema you define. You describe those entities once as a JSON Schema, submit it with the document, and the extraction assembles them across every page into a single result. Each value carries the page region it came from, so you can trace any field in the output back to its source. The work is entity-driven rather than page-driven, so the same schema runs against a three-page letter or a fifteen-hundred-page contract without change. Pages are processed in parallel, and references to the same entity on different pages land on the same place in the output.

Deep transform is a preview feature. The request and response shapes may change before general availability. The SDK helper methods shown here are available in the 2.0.4 Python and TypeScript SDKs.

Before you begin

You need three things to run an extraction:

An API key, sent in the Meibel-API-Key header. The SDKs read it from the MEIBEL_API_KEY environment variable in the examples below.
A JSON Schema describing the entities to extract. The shape of this schema is the shape of your output: objects become nested objects, arrays become collections, and scalar fields become single values. A field declared as a string comes back as a string; a field declared as an object comes back fully populated.
A document to extract from, such as a PDF.
Domain guidance (optional), sent as guidance. This is free-text instruction about how your domain names and structures things: the terminology it uses, how to disambiguate values that look alike, and which fields to prioritize. Specialized documents extract more accurately when the model is told the conventions the document assumes its reader already knows.

Submit an extraction

Submitting sends the document and schema together and returns a job ID right away, so the extraction runs server-side while your application stays responsive. The root_name names the top-level entity in your schema, and max_pages caps how many pages are processed, which is useful for a quick trial before a full run. Pass your domain instructions as guidance so the extraction reads the document the way a specialist in your field would.

import json
import os
from meibel import MeibelClient

client = MeibelClient(api_key=os.environ["MEIBEL_API_KEY"])

with open("schema.json") as f:
    schema = json.load(f)

with open("guidance.md") as f:
    guidance = f.read()

job = client.documents.submit_deep_transform(
    file="contract.pdf",
    schema=schema,
    root_name="contract",
    guidance=guidance,
)

print("Job submitted:", job.job_id)

import { readFile } from "node:fs/promises";
import { MeibelClient } from "meibel";

const client = new MeibelClient({ apiKey: process.env.MEIBEL_API_KEY });

const schema = JSON.parse(await readFile("schema.json", "utf8"));
const guidance = await readFile("guidance.md", "utf8");
const file = new File([await readFile("contract.pdf")], "contract.pdf", {
  type: "application/pdf",
});

const job = await client.documents.submitDeepTransform({
  file,
  schema,
  rootName: "contract",
  guidance,
});

console.log("Job submitted:", job.jobId);

curl -X POST https://api.meibel.ai/v2/documents/deep-transform \
  -H "Meibel-API-Key: $MEIBEL_API_KEY" \
  -F "file=@contract.pdf;type=application/pdf" \
  -F "schema=<schema.json" \
  -F "root_name=contract" \
  -F "guidance=<guidance.md"

Store the returned job ID to check status and download results. Submission is idempotent on the document and schema pair, so repeating the same request returns the same job rather than starting a new one. The examples read guidance from a guidance.md file, which keeps longer instructions out of your code. Guidance is optional, so you can omit it for documents that need no domain context, and it can grow from a sentence to a full page as a domain demands.

Check status

A deep transform runs asynchronously across the document’s pages, so you poll the job until it reaches a terminal state. The status moves through queued and running to either succeeded or failed.

import time

while True:
    status = client.documents.get_deep_transform_status(job.job_id)
    print("Status:", status.status)

    if status.status in ("succeeded", "failed"):
        break

    time.sleep(5)

let status;

do {
  status = await client.documents.getDeepTransformStatus(job.jobId);
  console.log("Status:", status.status);

  if (status.status === "succeeded" || status.status === "failed") {
    break;
  }

  await new Promise((resolve) => setTimeout(resolve, 5000));
} while (true);

curl https://api.meibel.ai/v2/documents/deep-transform/$JOB_ID \
  -H "Meibel-API-Key: $MEIBEL_API_KEY"

A succeeded job also reports metrics (such as wall-clock time and identity resolution rate) and an aeq extraction-quality score, alongside the list of artifacts you can download.

Download results

Once the job succeeds, the extraction is available as named artifacts. The output.json artifact holds the schema-conformant result. The provenance.json artifact maps each value in that result to its source spans, so a reviewer can land on the exact page region a field came from.

import json

result = client.documents.download_deep_transform_artifact(
    job.job_id, "output.json"
)

# The artifact is a parsed JSON object shaped like your schema.
print(json.dumps(result, indent=2))

const result = await client.documents.downloadDeepTransformArtifact(
  job.jobId,
  "output.json",
);

// The artifact is a parsed JSON object shaped like your schema.
console.log(JSON.stringify(result, null, 2));

curl https://api.meibel.ai/v2/documents/deep-transform/$JOB_ID/artifact/output.json \
  -H "Meibel-API-Key: $MEIBEL_API_KEY"

The result conforms to the schema you submitted, and values for the same entity are merged from every page they appear on. The nesting and field names mirror your schema exactly.

You can review provenance visually in the Meibel app instead of reading provenance.json by hand. Open the Transform menu in the app sidebar and enter your job ID, or go straight to https://app.meibel.ai/projects/{project_id}/transform/job/{job_id} with your project and job IDs substituted. Either way you see each extracted value linked to the page region it came from.

Reuse a parsed document

If you have already parsed a document through the Documents API, you can run a deep transform against that parse instead of uploading the file again. Submit to POST /documents/deep-transform/from-document with the parse job ID and your schema, and the document is not parsed a second time.

Prompts & Artifact Schemas

Define the JSON Schema that shapes your extraction output.

Document Processing

Parse a document first, then reuse that parse in a deep transform.

​Before you begin

​Submit an extraction

​Check status

​Download results

​Reuse a parsed document

Prompts & Artifact Schemas

Document Processing

Before you begin

Submit an extraction

Check status

Download results

Reuse a parsed document