# Extracting structured data from unstructured text, including with LLMs

How to turn free text into structured fields with schema-based extraction, JSON output, prompt design and field validation, including LLM-assisted workflows.

Extracting structured data from unstructured text means turning free-flowing writing into named fields a database or spreadsheet can hold. You define a schema, then pull entities and attributes out of the text to fill it. Large language models can do that extraction when a schema, a clear prompt and field validation guide them. 

- Start with the schema, because the fields you want decide how the text gets read.
- An LLM extractor needs an explicit prompt that names each field and its allowed values.
- Ask for JSON output and validate every field before you trust the result.
- Rule-based parsing is predictable and costs nothing per call, while LLMs handle messy wording.
- A review step catches wrong fields before they enter your shortlist.
 

## What structured extraction actually produces 

Structured extraction turns unstructured text into records that share one set of fields. A listing description becomes a row with a price field, a size field and a condition field. The output is data, not prose, so each value sits under a known label. 

The schema is the contract for that output, and it lists every field, its type and whether it is required. Schema-based extraction reads the text against that contract instead of gathering whatever looks interesting. A field the schema does not name has no place in the record. 

Entity and attribute extraction is the core operation inside the schema. An entity is the thing described, such as a product or a property. An attribute is a fact about it, such as material, weight or year of manufacture. 

## Why free text resists a simple table 

Unstructured text hides its facts inside sentences, and the same fact can appear in many wordings. One description states a size in centimeters, another implies it through a comparison. Text parsing rules written for one wording fail on the next. 

Missing and conflicting values are normal in free text. A description may omit the material entirely or mention two dimensions that disagree. Your extraction plan needs a way to mark a field as absent rather than guess a value. 

Units, abbreviations and qualifiers add noise to every field. A value of 'approx. 40' carries a number, a unit guess and a hedge all at once. Decide in the schema how much of that detail the field keeps. 

## Designing the schema before you prompt 

Schema design decides what the extractor can return, so settle the fields before you write any prompt. List the fields your shortlist needs side by side across listings. Give each field a type, such as text, number, date or a fixed set of choices. 

Keep the schema narrow enough to test and wide enough to be useful. A field that exists only for one listing should not sit in the shared schema. Required fields and optional fields belong in separate groups so a missing value is visible. 

Where a field has allowed values, write them into the schema as an enumeration. A condition field limited to new, used and for parts cannot drift into free wording. This restriction makes field validation much simpler later. 

## Writing a prompt that fills the schema 

Prompt design for extraction works best when the prompt repeats the schema and the output rules in plain words. The prompt should name each field, its type and what counts as a valid value. Instructions written once in the schema and once in the prompt reduce silent mismatches. 

Tell the model to return nothing for a field the text does not support. An invented value is harder to spot than an empty one. Ask for a short reason field when a value comes from an inference rather than a direct statement. 

Keep the source text separate from the instructions inside the prompt. A clear boundary stops the model from reading part of the listing as an instruction. Longer text may need splitting into chunks that each fit the model's context. 

## Getting reliable JSON output from a model 

JSON output gives you a predictable shape that a program can read without guessing. Large language models can produce JSON, yet a free-form reply may add commentary around it. Ask for one JSON object and nothing else in the response. 

A structured output mode or a JSON schema constraint tightens the reply further. When the tool supports it, the model can only emit keys the schema allows. Without that support, parse the reply and reject anything that fails the shape check. 

Fix the small format problems with a repair step rather than a rewrite of the prompt. Trimming surrounding text and checking brackets handles many failures. A second pass that returns the same content as clean JSON often resolves the rest. 

## Checking fields before they reach your list 

Field validation is the step that turns a raw extraction into data you can compare. Each field gets a rule: a number must parse, a choice must sit inside its list, a required field must be present. A record that fails a rule goes to review, not into the shortlist. 

Validation rules come straight from the schema and from the domain. A size field cannot be negative, and a year field cannot sit in the future. Cross-field checks catch contradictions such as a stated total that ignores a listed surcharge. 

Track where each value came from, whether direct text or inference, and keep that note beside the field. A reviewer can then check the weak values first. Confidence scores from the model help order the review but do not replace the rules. 

## Rule-based parsing or an LLM extractor 

Rule-based parsing and LLM extraction solve different parts of the same job. Patterns and regular expressions handle stable formats such as reference codes and labeled prices. A model handles the sentences that no pattern predicts. The table sets out the trade-offs to weigh for each approach. 

A hybrid pipeline often works well, with rules first and a model for the leftovers. Run the cheap deterministic checks, then send unmatched text to the model. Route every model result back through the same validation rules. 

Cost and repeatability decide the balance. Rules return the same answer every run, and a model can vary its wording between calls. Pin the model settings and store the raw reply for every record you keep. 
| Approach | Handles | Main risk | 
| Rule-based parsing | Stable formats and labeled fields | Breaks on new wording | 
| LLM extraction | Messy sentences and implied facts | Needs validation and review | 
| Hybrid pipeline | Both, in one workflow | More pieces to monitor |  

## Building a shortlist from extracted records 

A shortlist works when the extracted records share the same fields, so differences show up in a column. Fill each field from the extraction and leave blanks where the text gave nothing. Side-by-side comparison then shows gaps instead of hiding them. 

Bring your own requirements into the same table as extra columns. A budget ceiling or a required attribute becomes a column you check against each record. The comparison stays honest because every row is read the same way. 

Keep the source text linked to each record for a final check. When two listings look alike in the table, the original wording settles the choice. A trained specialist should confirm any field that carries safety or legal weight. 

## Questions about structured text extraction 

### Can an LLM extract structured data from text reliably? 

An LLM can extract structured data from text reliably when a schema and validation rules constrain it. The model handles varied wording, and the rules catch values that break the schema. Reliability comes from that pairing, not from the model alone. 

### How do I extract structured data from unstructured text? 

To extract structured data from unstructured text, define the fields you need and the type of each one. Then parse or prompt for those fields and validate every value against the schema. Keep the source text beside each record for review. 

### What is the difference between structured output and plain JSON output? 

Structured output is a mode where the model is constrained to a schema, while plain JSON output is any reply in JSON form. Structured output removes most shape errors before they happen. Plain JSON output still needs parsing and checking. 

### Which fields should my extraction schema include? 

Your extraction schema should include the fields you compare across listings plus any field that decides a shortlist. Give each field a type and, where possible, a fixed set of allowed values. Leave out fields that apply to only one listing. 

### When should I use rules instead of an LLM for extraction? 

Use rules instead of an LLM when the text format is stable and labeled, such as codes or fixed price patterns. Rules are repeatable and easy to test. Send the unpredictable wording to a model and validate its output the same way.
