google cloud platform – How to use DocumentAI to extract data and bring the results to BQ using BQML?

I built a custom extractor in Document AI.
Deployed version : pretrained-foundation-model-v1.3-2024-08-31

    # Create a remote model to register your Doc AI processor in BigQuery.
CREATE OR REPLACE MODEL `data-dev.iiv.invoice_parse_model`
REMOTE WITH CONNECTION `data-dev.eu.doc_conn`
OPTIONS (
  remote_service_type="CLOUD_AI_DOCUMENT_V1",   
  document_processor="projects/2887432/locations/eu/processors/e97de02/processorVersions/pretrained-foundation-model-v1.3-2024-08-31"
);

# Create an object table in BigQuery that maps to the document files stored in Cloud Storage.
CREATE OR REPLACE EXTERNAL TABLE `data-dev.iiv.invoice_object_table`
WITH CONNECTION `data-dev.eu.doc_conn`
OPTIONS (
  object_metadata="SIMPLE",
  uris = ['gs://iiv/*'] 
);

# Parse documents using ml.process_document 
 CREATE OR REPLACE TABLE `data-dev.iiv.invoice_parsed_table`
 AS 
 SELECT * 
 FROM ML.PROCESS_DOCUMENT(
  MODEL `data-dev.iiv.invoice_parse_model`,
  TABLE `data-dev.iiv.invoice_object_table`
 )
 WHERE content_type="application/pdf";

Now, I am expecting a schema with the following fields:
amount, description, quantity and units.

But, first – I do not have any schema defined when i click on invoice_parse_model.
And secondly, instead of amount, description, quantity and units… I simply get the schema of invoice_object_table.

Here are a few screenshots, of how the output looks like & the cell value of ml_process_document_result.

—– the first ss
enter image description here

—– the second ss
enter image description here

I have validated the connector values and model specifications.

I want the table to have the parsed values per field defined in the training schema.

Read more here: Source link