google cloud platform – How to use DocumentAI to extract data and bring the results to BQ using BQML?
I built a custom extractor in Document AI.
Deployed version : pretrained-foundation-model-v1.3-2024-08-31
# Create a remote model to register your Doc AI processor in BigQuery.
CREATE OR REPLACE MODEL `data-dev.iiv.invoice_parse_model`
REMOTE WITH CONNECTION `data-dev.eu.doc_conn`
OPTIONS (
remote_service_type="CLOUD_AI_DOCUMENT_V1",
document_processor="projects/2887432/locations/eu/processors/e97de02/processorVersions/pretrained-foundation-model-v1.3-2024-08-31"
);
# Create an object table in BigQuery that maps to the document files stored in Cloud Storage.
CREATE OR REPLACE EXTERNAL TABLE `data-dev.iiv.invoice_object_table`
WITH CONNECTION `data-dev.eu.doc_conn`
OPTIONS (
object_metadata="SIMPLE",
uris = ['gs://iiv/*']
);
# Parse documents using ml.process_document
CREATE OR REPLACE TABLE `data-dev.iiv.invoice_parsed_table`
AS
SELECT *
FROM ML.PROCESS_DOCUMENT(
MODEL `data-dev.iiv.invoice_parse_model`,
TABLE `data-dev.iiv.invoice_object_table`
)
WHERE content_type="application/pdf";
Now, I am expecting a schema with the following fields:
amount, description, quantity and units.
But, first – I do not have any schema defined when i click on invoice_parse_model.
And secondly, instead of amount, description, quantity and units… I simply get the schema of invoice_object_table.
Here are a few screenshots, of how the output looks like & the cell value of ml_process_document_result.
I have validated the connector values and model specifications.
I want the table to have the parsed values per field defined in the training schema.
Read more here: Source link


